{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a623161a",
   "metadata": {},
   "source": [
    "# ASSISTments2017 Data Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1acacbe8",
   "metadata": {},
   "source": [
    "## Data Description\n",
    "\n",
    "### Column Description\n",
    "\n",
    "\n",
    "| Field    | Annotation                                          |\n",
    "| :- | :- |\n",
    "| student id | \ta deidentified ID/tag used for identifying an individual student |\n",
    "| SY ASSISTments Usage | the academic years the student used ASSISTments |\n",
    "| AveKnow |  average student knowledge level (according to Bayesian Knowledge Tracing algorithm -- cf. Corbett & Anderson, 1995) |\n",
    "| AveCarelessness | average student carelessness (according to San Pedro, Baker, & Rodrigo, 2011 model) |\n",
    "| AveCorrect | average student correctness |\n",
    "| NumActions | total number of student actions in system |\n",
    "| AveResBored |  average student affect: boredom (see Pardos, Baker, San Pedro, Gowda, & Gowda, 2014) |\n",
    "| AveResEngcon | average student affect:engaged concentration (see Pardos, Baker, San Pedro, Gowda, & Gowda, 2014)|\n",
    "| AveResConf | average student affect:confusion (see Pardos, Baker, San Pedro, Gowda, & Gowda, 2014) |\n",
    "| AveResFrust | average student affect:frustration (see Pardos, Baker, San Pedro, Gowda, & Gowda, 2014) |\n",
    "| AveResOfftask | average student affect: off task (see Pardos, Baker, San Pedro, Gowda, & Gowda, 2014 and also Baker, 2007) |\n",
    "| AveResGaming | average student affect:gaming the system  (see Pardos, Baker, San Pedro, Gowda, & Gowda, 2014 and also Baker Corbett Koedinger & Wagner, 2004) |\n",
    "| actionId | the unique id of this specific action | \n",
    "| skill |  a tag used for identifying the cognitive skill related to the problem (see Razzaq, Heffernan, Feng, & Pardos, 2007) |\n",
    "| problemId | a unique ID used for identifying a single problem |\n",
    "| assignmentId | a unique ID used for identifying an assignment |\n",
    "| assistmentId |  a unique ID used for identifying an assistment (a instance of a multi-part problem) |\n",
    "| startTime | when did the student start the problem (UNIX time, seconds) |\n",
    "|  endTime | when did the student end the problem (UNIX time, seconds) |\n",
    "| timeTaken | Time spent on the current step |\n",
    "| correct | Answer is correct |\n",
    "| original | Problem is original not a scaffolding problem |\n",
    "|  hint | Action is a hint response |\n",
    "| hintCount | Total number of hints requested so far |\n",
    "| hintTotal | total number of hints requested for the problem |\n",
    "| scaffold | Problem is a scaffolding problem |\n",
    "| bottomHint | Bottom-out hint is used |\n",
    "| attemptCount | Total problems attempted in the tutor so far. |\n",
    "| problemType | the type of the problem |\n",
    "| frIsHelpRequest | First response is a help request |\n",
    "| frPast5HelpRequest | Number of last 5 First responses that included a help request |\n",
    "| frPast8HelpRequest | Number of last 8 First responses that included a help request |\n",
    "| stlHintUsed | Second to last hint is used an indicates a hint that gives considerable detail but is not quite bottom-out |\n",
    "| past8BottomOut | Number of last 8 problems that used the bottom-out hint. |\n",
    "| totalFrPercentPastWrong | Percent of all past problems that were wrong on this KC. |\n",
    "| totalFrPastWrongCount | Total first responses wrong attempts in the tutor so far. |\n",
    "| frPast5WrongCount | Number of last 5 First responses that were wrong |\n",
    "| frPast8WrongCount | Number of last 8 First responses that were wrong |\n",
    "| totalFrTimeOnSkill | Total first response time spent on this KC across all problems |\n",
    "| timeSinceSkill | Time since the current KC was last seen. |\n",
    "| frWorkingInSchool | First response Working during school hours (between 7:00 am and 3:00 pm) |\n",
    "| totalFrAttempted | Total first responses attempted in the tutor so far. |\n",
    "| totalFrSkillOpportunities | Total first response practice opportunities on this KC so far. |\n",
    "| responseIsFillIn | Response is filled in (No list of answers available) | \n",
    "| responseIsChosen | Response is chosen from a list of answers (Multiple choice, etc). |\n",
    "| endsWithScaffolding | Problem ends with scaffolding |\n",
    "| endsWithAutoScaffolding  | Problem ends with automatic scaffolding |\n",
    "| frTimeTakenOnScaffolding | First response time taken on scaffolding problems |\n",
    "| frTotalSkillOpportunitiesScaffolding | Total first response practice opportunities on this skill so far |\n",
    "| totalFrSkillOpportunitiesByScaffolding | Total first response scaffolding opportunities for this KC so far |\n",
    "| frIsHelpRequestScaffolding | First response is a help request Scaffolding |\n",
    "| timeGreater5Secprev2wrong | Long pauses after 2 Consecutive wrong answers |\n",
    "| sumRight | NaN |\n",
    "| helpAccessUnder2Sec | Time spent on help was under 2 seconds |\n",
    "| timeGreater10SecAndNextActionRight | Long pause after correct answer |\n",
    "| consecutiveErrorsInRow | Total number of 2 wrong answers in a row across all the problems |\n",
    "| sumTime3SDWhen3RowRight | NaN |\n",
    "| sumTimePerSkill | NaN |\n",
    "| totalTimeByPercentCorrectForskill | Total time spent on this KC across all problems divided by percent correct for the same KC |\n",
    "| prev5count | NaN |\n",
    "| timeOver80 | NaN |\n",
    "| manywrong | NaN |\n",
    "| confidence(BORED) | the confidence of the student affect prediction: bored |\n",
    "| confidence(CONCENTRATING) | the confidence of the student affect prediction: concecntrating |\n",
    "| confidence(CONFUSED) | the confidence of the student affect prediction: confused |\n",
    "| confidence(FRUSTRATED) | the confidence of the student affect prediction: frustrated  |\n",
    "| confidence(OFF TASK) | the confidence of the student affect prediction: off task |\n",
    "| confidence(GAMING) | the confidence of the student affect prediction: gaming |\n",
    "| RES_BORED | rescaled of the confidence of the student affect prediction: boredom |\n",
    "| RES_CONCENTRATING | rescaled of the confidence of the student affect prediction: concentration |\n",
    "| RES_CONFUSED | rescaled of the confidence of the student affect prediction: confusion |\n",
    "| RES_FRUSTRATED | rescaled of the confidence of the student affect prediction: frustration |\n",
    "| RES_OFFTASK | rescaled of the confidence of the student affect prediction: off task |\n",
    "| RES_GAMING | rescaled of the confidence of the student affect prediction: gaming |\n",
    "| Ln-1 | baysian knowledge tracing's knowledge estimate at the previous time step |\n",
    "| Ln | baysian knowledge tracing's knowledge estimate at the time step |\n",
    "| schoolID | the id (anonymized) of the school the student was in during the year the data was collected |\n",
    "| MCAS | Massachusetts Comprehensive Assessment System test score. In short, this number is the student's state test score (outside ASSISTments) during that year. -999 represents the data is missing |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "1d4da69e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import plotly.express as asz\n",
    "from plotly.subplots import make_subplots\n",
    "import plotly.graph_objs as go"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "e17d8908",
   "metadata": {},
   "outputs": [],
   "source": [
    "path = \"anonymized_full_release_competition_dataset.csv\"\n",
    "data = pd.read_csv(path, encoding = \"ISO-8859-15\",low_memory=False) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "4036aaa8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>studentId</th>\n",
       "      <th>MiddleSchoolId</th>\n",
       "      <th>InferredGender</th>\n",
       "      <th>SY ASSISTments Usage</th>\n",
       "      <th>AveKnow</th>\n",
       "      <th>AveCarelessness</th>\n",
       "      <th>AveCorrect</th>\n",
       "      <th>NumActions</th>\n",
       "      <th>AveResBored</th>\n",
       "      <th>AveResEngcon</th>\n",
       "      <th>AveResConf</th>\n",
       "      <th>AveResFrust</th>\n",
       "      <th>AveResOfftask</th>\n",
       "      <th>AveResGaming</th>\n",
       "      <th>action_num</th>\n",
       "      <th>skill</th>\n",
       "      <th>problemId</th>\n",
       "      <th>problemType</th>\n",
       "      <th>assignmentId</th>\n",
       "      <th>assistmentId</th>\n",
       "      <th>startTime</th>\n",
       "      <th>endTime</th>\n",
       "      <th>timeTaken</th>\n",
       "      <th>correct</th>\n",
       "      <th>original</th>\n",
       "      <th>hint</th>\n",
       "      <th>hintCount</th>\n",
       "      <th>hintTotal</th>\n",
       "      <th>scaffold</th>\n",
       "      <th>bottomHint</th>\n",
       "      <th>attemptCount</th>\n",
       "      <th>frIsHelpRequest</th>\n",
       "      <th>frPast5HelpRequest</th>\n",
       "      <th>frPast8HelpRequest</th>\n",
       "      <th>stlHintUsed</th>\n",
       "      <th>past8BottomOut</th>\n",
       "      <th>totalFrPercentPastWrong</th>\n",
       "      <th>totalFrPastWrongCount</th>\n",
       "      <th>frPast5WrongCount</th>\n",
       "      <th>frPast8WrongCount</th>\n",
       "      <th>totalFrTimeOnSkill</th>\n",
       "      <th>timeSinceSkill</th>\n",
       "      <th>frWorkingInSchool</th>\n",
       "      <th>totalFrAttempted</th>\n",
       "      <th>totalFrSkillOpportunities</th>\n",
       "      <th>responseIsFillIn</th>\n",
       "      <th>responseIsChosen</th>\n",
       "      <th>endsWithScaffolding</th>\n",
       "      <th>endsWithAutoScaffolding</th>\n",
       "      <th>frTimeTakenOnScaffolding</th>\n",
       "      <th>frTotalSkillOpportunitiesScaffolding</th>\n",
       "      <th>totalFrSkillOpportunitiesByScaffolding</th>\n",
       "      <th>frIsHelpRequestScaffolding</th>\n",
       "      <th>timeGreater5Secprev2wrong</th>\n",
       "      <th>sumRight</th>\n",
       "      <th>helpAccessUnder2Sec</th>\n",
       "      <th>timeGreater10SecAndNextActionRight</th>\n",
       "      <th>consecutiveErrorsInRow</th>\n",
       "      <th>sumTime3SDWhen3RowRight</th>\n",
       "      <th>sumTimePerSkill</th>\n",
       "      <th>totalTimeByPercentCorrectForskill</th>\n",
       "      <th>Prev5count</th>\n",
       "      <th>timeOver80</th>\n",
       "      <th>manywrong</th>\n",
       "      <th>confidence(BORED)</th>\n",
       "      <th>confidence(CONCENTRATING)</th>\n",
       "      <th>confidence(CONFUSED)</th>\n",
       "      <th>confidence(FRUSTRATED)</th>\n",
       "      <th>confidence(OFF TASK)</th>\n",
       "      <th>confidence(GAMING)</th>\n",
       "      <th>RES_BORED</th>\n",
       "      <th>RES_CONCENTRATING</th>\n",
       "      <th>RES_CONFUSED</th>\n",
       "      <th>RES_FRUSTRATED</th>\n",
       "      <th>RES_OFFTASK</th>\n",
       "      <th>RES_GAMING</th>\n",
       "      <th>Ln-1</th>\n",
       "      <th>Ln</th>\n",
       "      <th>MCAS</th>\n",
       "      <th>Enrolled</th>\n",
       "      <th>Selective</th>\n",
       "      <th>isSTEM</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>Male</td>\n",
       "      <td>2004-2005</td>\n",
       "      <td>0.352416</td>\n",
       "      <td>0.183276</td>\n",
       "      <td>0.483902</td>\n",
       "      <td>1056</td>\n",
       "      <td>0.208389</td>\n",
       "      <td>0.679126</td>\n",
       "      <td>0.115905</td>\n",
       "      <td>0.112408</td>\n",
       "      <td>0.156503</td>\n",
       "      <td>0.196561</td>\n",
       "      <td>9950</td>\n",
       "      <td>properties-of-geometric-figures</td>\n",
       "      <td>1118</td>\n",
       "      <td>textfieldquestion</td>\n",
       "      <td>20405010</td>\n",
       "      <td>104051118</td>\n",
       "      <td>1096470301</td>\n",
       "      <td>1096470350</td>\n",
       "      <td>49.0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.597865</td>\n",
       "      <td>0.234294</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.838710</td>\n",
       "      <td>0.008522</td>\n",
       "      <td>0.376427</td>\n",
       "      <td>0.320317</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.785585</td>\n",
       "      <td>0.000264</td>\n",
       "      <td>0.13</td>\n",
       "      <td>0.061190409</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>Male</td>\n",
       "      <td>2004-2005</td>\n",
       "      <td>0.352416</td>\n",
       "      <td>0.183276</td>\n",
       "      <td>0.483902</td>\n",
       "      <td>1056</td>\n",
       "      <td>0.208389</td>\n",
       "      <td>0.679126</td>\n",
       "      <td>0.115905</td>\n",
       "      <td>0.112408</td>\n",
       "      <td>0.156503</td>\n",
       "      <td>0.196561</td>\n",
       "      <td>9951</td>\n",
       "      <td>properties-of-geometric-figures</td>\n",
       "      <td>1119</td>\n",
       "      <td>noprobtype</td>\n",
       "      <td>20405010</td>\n",
       "      <td>104051119</td>\n",
       "      <td>1096470350</td>\n",
       "      <td>1096470354</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>49.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>53.0</td>\n",
       "      <td>106.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.355694</td>\n",
       "      <td>0.992585</td>\n",
       "      <td>0.9375</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.600000</td>\n",
       "      <td>0.047821</td>\n",
       "      <td>0.156027</td>\n",
       "      <td>0.995053</td>\n",
       "      <td>0.887452</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.468252</td>\n",
       "      <td>0.001483</td>\n",
       "      <td>0.061190409</td>\n",
       "      <td>0.213509945</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>Male</td>\n",
       "      <td>2004-2005</td>\n",
       "      <td>0.352416</td>\n",
       "      <td>0.183276</td>\n",
       "      <td>0.483902</td>\n",
       "      <td>1056</td>\n",
       "      <td>0.208389</td>\n",
       "      <td>0.679126</td>\n",
       "      <td>0.115905</td>\n",
       "      <td>0.112408</td>\n",
       "      <td>0.156503</td>\n",
       "      <td>0.196561</td>\n",
       "      <td>9952</td>\n",
       "      <td>sum-of-interior-angles-more-than-3-sides</td>\n",
       "      <td>1120</td>\n",
       "      <td>noprobtype</td>\n",
       "      <td>20405010</td>\n",
       "      <td>104051120</td>\n",
       "      <td>1096470354</td>\n",
       "      <td>1096470360</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.355694</td>\n",
       "      <td>0.992585</td>\n",
       "      <td>0.9375</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.600000</td>\n",
       "      <td>0.047821</td>\n",
       "      <td>0.156027</td>\n",
       "      <td>0.995053</td>\n",
       "      <td>0.887452</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.468252</td>\n",
       "      <td>0.001483</td>\n",
       "      <td>0.116</td>\n",
       "      <td>0.033305768</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>Male</td>\n",
       "      <td>2004-2005</td>\n",
       "      <td>0.352416</td>\n",
       "      <td>0.183276</td>\n",
       "      <td>0.483902</td>\n",
       "      <td>1056</td>\n",
       "      <td>0.208389</td>\n",
       "      <td>0.679126</td>\n",
       "      <td>0.115905</td>\n",
       "      <td>0.112408</td>\n",
       "      <td>0.156503</td>\n",
       "      <td>0.196561</td>\n",
       "      <td>9953</td>\n",
       "      <td>sum-of-interior-angles-more-than-3-sides</td>\n",
       "      <td>1120</td>\n",
       "      <td>noprobtype</td>\n",
       "      <td>20405010</td>\n",
       "      <td>104051120</td>\n",
       "      <td>1096470360</td>\n",
       "      <td>1096470378</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.355694</td>\n",
       "      <td>0.617065</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.204082</td>\n",
       "      <td>0.343996</td>\n",
       "      <td>0.156027</td>\n",
       "      <td>0.744520</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.108417</td>\n",
       "      <td>0.010665</td>\n",
       "      <td>0.116</td>\n",
       "      <td>0.033305768</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>2</td>\n",
       "      <td>Male</td>\n",
       "      <td>2004-2005</td>\n",
       "      <td>0.352416</td>\n",
       "      <td>0.183276</td>\n",
       "      <td>0.483902</td>\n",
       "      <td>1056</td>\n",
       "      <td>0.208389</td>\n",
       "      <td>0.679126</td>\n",
       "      <td>0.115905</td>\n",
       "      <td>0.112408</td>\n",
       "      <td>0.156503</td>\n",
       "      <td>0.196561</td>\n",
       "      <td>9954</td>\n",
       "      <td>sum-of-interior-angles-more-than-3-sides</td>\n",
       "      <td>1121</td>\n",
       "      <td>noprobtype</td>\n",
       "      <td>20405010</td>\n",
       "      <td>104051121</td>\n",
       "      <td>1096470378</td>\n",
       "      <td>1096470380</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>26.0</td>\n",
       "      <td>77.999999</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.355694</td>\n",
       "      <td>0.617065</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.204082</td>\n",
       "      <td>0.343996</td>\n",
       "      <td>0.156027</td>\n",
       "      <td>0.744520</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.108417</td>\n",
       "      <td>0.010665</td>\n",
       "      <td>0.033305768</td>\n",
       "      <td>0.118385889</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   studentId  MiddleSchoolId InferredGender SY ASSISTments Usage   AveKnow  \\\n",
       "0          8               2           Male            2004-2005  0.352416   \n",
       "1          8               2           Male            2004-2005  0.352416   \n",
       "2          8               2           Male            2004-2005  0.352416   \n",
       "3          8               2           Male            2004-2005  0.352416   \n",
       "4          8               2           Male            2004-2005  0.352416   \n",
       "\n",
       "   AveCarelessness  AveCorrect  NumActions  AveResBored  AveResEngcon  \\\n",
       "0         0.183276    0.483902        1056     0.208389      0.679126   \n",
       "1         0.183276    0.483902        1056     0.208389      0.679126   \n",
       "2         0.183276    0.483902        1056     0.208389      0.679126   \n",
       "3         0.183276    0.483902        1056     0.208389      0.679126   \n",
       "4         0.183276    0.483902        1056     0.208389      0.679126   \n",
       "\n",
       "   AveResConf  AveResFrust  AveResOfftask  AveResGaming  action_num  \\\n",
       "0    0.115905     0.112408       0.156503      0.196561        9950   \n",
       "1    0.115905     0.112408       0.156503      0.196561        9951   \n",
       "2    0.115905     0.112408       0.156503      0.196561        9952   \n",
       "3    0.115905     0.112408       0.156503      0.196561        9953   \n",
       "4    0.115905     0.112408       0.156503      0.196561        9954   \n",
       "\n",
       "                                      skill  problemId        problemType  \\\n",
       "0           properties-of-geometric-figures       1118  textfieldquestion   \n",
       "1           properties-of-geometric-figures       1119         noprobtype   \n",
       "2  sum-of-interior-angles-more-than-3-sides       1120         noprobtype   \n",
       "3  sum-of-interior-angles-more-than-3-sides       1120         noprobtype   \n",
       "4  sum-of-interior-angles-more-than-3-sides       1121         noprobtype   \n",
       "\n",
       "   assignmentId  assistmentId   startTime     endTime  timeTaken  correct  \\\n",
       "0      20405010     104051118  1096470301  1096470350       49.0        0   \n",
       "1      20405010     104051119  1096470350  1096470354        4.0        1   \n",
       "2      20405010     104051120  1096470354  1096470360        6.0        0   \n",
       "3      20405010     104051120  1096470360  1096470378       18.0        0   \n",
       "4      20405010     104051121  1096470378  1096470380        2.0        1   \n",
       "\n",
       "   original  hint  hintCount  hintTotal  scaffold  bottomHint  attemptCount  \\\n",
       "0         1     1          1          1         0           0             1   \n",
       "1         0     0          0          0         1           0             1   \n",
       "2         0     0          0          0         0           0             1   \n",
       "3         0     0          0          0         0           0             2   \n",
       "4         0     0          0          1         0           0             1   \n",
       "\n",
       "   frIsHelpRequest  frPast5HelpRequest  frPast8HelpRequest  stlHintUsed  \\\n",
       "0                1                   0                   0            0   \n",
       "1                1                   1                   1            0   \n",
       "2                0                   0                   0            0   \n",
       "3                0                   0                   0            0   \n",
       "4                0                   0                   0            0   \n",
       "\n",
       "   past8BottomOut  totalFrPercentPastWrong  totalFrPastWrongCount  \\\n",
       "0               0                      0.0                      0   \n",
       "1               0                      0.0                      0   \n",
       "2               0                      0.0                      0   \n",
       "3               0                      0.0                      1   \n",
       "4               0                      1.0                      1   \n",
       "\n",
       "   frPast5WrongCount  frPast8WrongCount  totalFrTimeOnSkill  timeSinceSkill  \\\n",
       "0                  0                  0                 0.0             0.0   \n",
       "1                  0                  0                49.0             0.0   \n",
       "2                  0                  0                 0.0             0.0   \n",
       "3                  0                  0                 0.0             0.0   \n",
       "4                  1                  1                 6.0             0.0   \n",
       "\n",
       "   frWorkingInSchool  totalFrAttempted  totalFrSkillOpportunities  \\\n",
       "0                  1                 0                          0   \n",
       "1                  1                 1                          1   \n",
       "2                  1                 2                          0   \n",
       "3                  1                 3                          1   \n",
       "4                  1                 3                          1   \n",
       "\n",
       "   responseIsFillIn  responseIsChosen  endsWithScaffolding  \\\n",
       "0                 0                 0                    0   \n",
       "1                 0                 0                    1   \n",
       "2                 0                 0                    0   \n",
       "3                 0                 0                    0   \n",
       "4                 0                 0                    0   \n",
       "\n",
       "   endsWithAutoScaffolding  frTimeTakenOnScaffolding  \\\n",
       "0                        0                       0.0   \n",
       "1                        0                       4.0   \n",
       "2                        0                       6.0   \n",
       "3                        0                       6.0   \n",
       "4                        0                       2.0   \n",
       "\n",
       "   frTotalSkillOpportunitiesScaffolding  \\\n",
       "0                                     0   \n",
       "1                                     0   \n",
       "2                                     0   \n",
       "3                                     1   \n",
       "4                                     1   \n",
       "\n",
       "   totalFrSkillOpportunitiesByScaffolding  frIsHelpRequestScaffolding  \\\n",
       "0                                     0.0                           0   \n",
       "1                                     0.0                           1   \n",
       "2                                     0.0                           0   \n",
       "3                                     0.0                           0   \n",
       "4                                     1.0                           0   \n",
       "\n",
       "   timeGreater5Secprev2wrong  sumRight  helpAccessUnder2Sec  \\\n",
       "0                          0         0                    0   \n",
       "1                          0         1                    0   \n",
       "2                          0         1                    0   \n",
       "3                          0         1                    0   \n",
       "4                          0         2                    0   \n",
       "\n",
       "   timeGreater10SecAndNextActionRight  consecutiveErrorsInRow  \\\n",
       "0                                   0                       0   \n",
       "1                                   1                       0   \n",
       "2                                   0                       0   \n",
       "3                                   0                       1   \n",
       "4                                   1                       0   \n",
       "\n",
       "   sumTime3SDWhen3RowRight  sumTimePerSkill  \\\n",
       "0                      0.0             49.0   \n",
       "1                      0.0             53.0   \n",
       "2                      0.0              6.0   \n",
       "3                      0.0             24.0   \n",
       "4                      0.0             26.0   \n",
       "\n",
       "   totalTimeByPercentCorrectForskill  Prev5count  timeOver80  manywrong  \\\n",
       "0                           0.000000           0           0          0   \n",
       "1                         106.000000           1           0          0   \n",
       "2                           0.000000           2           0          0   \n",
       "3                           0.000000           3           0          0   \n",
       "4                          77.999999           4           0          1   \n",
       "\n",
       "   confidence(BORED)  confidence(CONCENTRATING)  confidence(CONFUSED)  \\\n",
       "0           0.597865                   0.234294                0.0000   \n",
       "1           0.355694                   0.992585                0.9375   \n",
       "2           0.355694                   0.992585                0.9375   \n",
       "3           0.355694                   0.617065                0.0000   \n",
       "4           0.355694                   0.617065                0.0000   \n",
       "\n",
       "   confidence(FRUSTRATED)  confidence(OFF TASK)  confidence(GAMING)  \\\n",
       "0                     0.0              0.838710            0.008522   \n",
       "1                     0.0              0.600000            0.047821   \n",
       "2                     0.0              0.600000            0.047821   \n",
       "3                     0.0              0.204082            0.343996   \n",
       "4                     0.0              0.204082            0.343996   \n",
       "\n",
       "   RES_BORED  RES_CONCENTRATING  RES_CONFUSED  RES_FRUSTRATED  RES_OFFTASK  \\\n",
       "0   0.376427           0.320317      0.000000             0.0     0.785585   \n",
       "1   0.156027           0.995053      0.887452             0.0     0.468252   \n",
       "2   0.156027           0.995053      0.887452             0.0     0.468252   \n",
       "3   0.156027           0.744520      0.000000             0.0     0.108417   \n",
       "4   0.156027           0.744520      0.000000             0.0     0.108417   \n",
       "\n",
       "   RES_GAMING         Ln-1           Ln  MCAS  Enrolled  Selective  isSTEM  \n",
       "0    0.000264         0.13  0.061190409    45         0          0     NaN  \n",
       "1    0.001483  0.061190409  0.213509945    45         0          0     NaN  \n",
       "2    0.001483        0.116  0.033305768    45         0          0     NaN  \n",
       "3    0.010665        0.116  0.033305768    45         0          0     NaN  \n",
       "4    0.010665  0.033305768  0.118385889    45         0          0     NaN  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.set_option('display.max_columns', 500)\n",
    "data.head() "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e82e7623",
   "metadata": {},
   "source": [
    "----\n",
    "## General features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "471df12c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>studentId</th>\n",
       "      <th>MiddleSchoolId</th>\n",
       "      <th>AveKnow</th>\n",
       "      <th>AveCarelessness</th>\n",
       "      <th>AveCorrect</th>\n",
       "      <th>NumActions</th>\n",
       "      <th>AveResBored</th>\n",
       "      <th>AveResEngcon</th>\n",
       "      <th>AveResConf</th>\n",
       "      <th>AveResFrust</th>\n",
       "      <th>AveResOfftask</th>\n",
       "      <th>AveResGaming</th>\n",
       "      <th>action_num</th>\n",
       "      <th>problemId</th>\n",
       "      <th>assignmentId</th>\n",
       "      <th>assistmentId</th>\n",
       "      <th>startTime</th>\n",
       "      <th>endTime</th>\n",
       "      <th>timeTaken</th>\n",
       "      <th>correct</th>\n",
       "      <th>original</th>\n",
       "      <th>hint</th>\n",
       "      <th>hintCount</th>\n",
       "      <th>hintTotal</th>\n",
       "      <th>scaffold</th>\n",
       "      <th>bottomHint</th>\n",
       "      <th>attemptCount</th>\n",
       "      <th>frIsHelpRequest</th>\n",
       "      <th>frPast5HelpRequest</th>\n",
       "      <th>frPast8HelpRequest</th>\n",
       "      <th>stlHintUsed</th>\n",
       "      <th>past8BottomOut</th>\n",
       "      <th>totalFrPercentPastWrong</th>\n",
       "      <th>totalFrPastWrongCount</th>\n",
       "      <th>frPast5WrongCount</th>\n",
       "      <th>frPast8WrongCount</th>\n",
       "      <th>totalFrTimeOnSkill</th>\n",
       "      <th>timeSinceSkill</th>\n",
       "      <th>frWorkingInSchool</th>\n",
       "      <th>totalFrAttempted</th>\n",
       "      <th>totalFrSkillOpportunities</th>\n",
       "      <th>responseIsFillIn</th>\n",
       "      <th>responseIsChosen</th>\n",
       "      <th>endsWithScaffolding</th>\n",
       "      <th>endsWithAutoScaffolding</th>\n",
       "      <th>frTimeTakenOnScaffolding</th>\n",
       "      <th>frTotalSkillOpportunitiesScaffolding</th>\n",
       "      <th>totalFrSkillOpportunitiesByScaffolding</th>\n",
       "      <th>frIsHelpRequestScaffolding</th>\n",
       "      <th>timeGreater5Secprev2wrong</th>\n",
       "      <th>sumRight</th>\n",
       "      <th>helpAccessUnder2Sec</th>\n",
       "      <th>timeGreater10SecAndNextActionRight</th>\n",
       "      <th>consecutiveErrorsInRow</th>\n",
       "      <th>sumTime3SDWhen3RowRight</th>\n",
       "      <th>sumTimePerSkill</th>\n",
       "      <th>totalTimeByPercentCorrectForskill</th>\n",
       "      <th>Prev5count</th>\n",
       "      <th>timeOver80</th>\n",
       "      <th>manywrong</th>\n",
       "      <th>confidence(BORED)</th>\n",
       "      <th>confidence(CONCENTRATING)</th>\n",
       "      <th>confidence(CONFUSED)</th>\n",
       "      <th>confidence(FRUSTRATED)</th>\n",
       "      <th>confidence(OFF TASK)</th>\n",
       "      <th>confidence(GAMING)</th>\n",
       "      <th>RES_BORED</th>\n",
       "      <th>RES_CONCENTRATING</th>\n",
       "      <th>RES_CONFUSED</th>\n",
       "      <th>RES_FRUSTRATED</th>\n",
       "      <th>RES_OFFTASK</th>\n",
       "      <th>RES_GAMING</th>\n",
       "      <th>MCAS</th>\n",
       "      <th>Enrolled</th>\n",
       "      <th>Selective</th>\n",
       "      <th>isSTEM</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>9.428160e+05</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>9.428160e+05</td>\n",
       "      <td>9.428160e+05</td>\n",
       "      <td>9.428160e+05</td>\n",
       "      <td>9.428160e+05</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>9.428160e+05</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.0</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942731.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>9.428160e+05</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>9.428160e+05</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>942816.000000</td>\n",
       "      <td>316974.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3844.844105</td>\n",
       "      <td>2.515472</td>\n",
       "      <td>0.195155</td>\n",
       "      <td>0.109436</td>\n",
       "      <td>0.372681</td>\n",
       "      <td>869.850594</td>\n",
       "      <td>0.232949</td>\n",
       "      <td>0.658442</td>\n",
       "      <td>0.098940</td>\n",
       "      <td>0.131406</td>\n",
       "      <td>0.172212</td>\n",
       "      <td>0.192703</td>\n",
       "      <td>1.849329e+06</td>\n",
       "      <td>1899.719319</td>\n",
       "      <td>1.198773e+07</td>\n",
       "      <td>6.061572e+07</td>\n",
       "      <td>1.120793e+09</td>\n",
       "      <td>1.120793e+09</td>\n",
       "      <td>29.747869</td>\n",
       "      <td>0.372681</td>\n",
       "      <td>0.264214</td>\n",
       "      <td>0.331025</td>\n",
       "      <td>1.218490</td>\n",
       "      <td>1.953967</td>\n",
       "      <td>0.385732</td>\n",
       "      <td>0.062794</td>\n",
       "      <td>2.673605</td>\n",
       "      <td>0.268104</td>\n",
       "      <td>1.947322</td>\n",
       "      <td>2.575323</td>\n",
       "      <td>0.004012</td>\n",
       "      <td>0.241678</td>\n",
       "      <td>0.227882</td>\n",
       "      <td>1.988008</td>\n",
       "      <td>0.719380</td>\n",
       "      <td>0.944750</td>\n",
       "      <td>376.213405</td>\n",
       "      <td>4.850802e+05</td>\n",
       "      <td>0.974588</td>\n",
       "      <td>193.316005</td>\n",
       "      <td>8.381019</td>\n",
       "      <td>0.023131</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.636085</td>\n",
       "      <td>0.005724</td>\n",
       "      <td>24.060853</td>\n",
       "      <td>3.989371</td>\n",
       "      <td>1.036189</td>\n",
       "      <td>0.670047</td>\n",
       "      <td>0.045388</td>\n",
       "      <td>145.982059</td>\n",
       "      <td>0.055674</td>\n",
       "      <td>0.207414</td>\n",
       "      <td>0.155307</td>\n",
       "      <td>0.087042</td>\n",
       "      <td>601.665586</td>\n",
       "      <td>2166.143744</td>\n",
       "      <td>4.972689</td>\n",
       "      <td>0.081323</td>\n",
       "      <td>0.715638</td>\n",
       "      <td>0.436958</td>\n",
       "      <td>5.400894e-01</td>\n",
       "      <td>0.134450</td>\n",
       "      <td>0.164114</td>\n",
       "      <td>0.256006</td>\n",
       "      <td>0.337888</td>\n",
       "      <td>0.232949</td>\n",
       "      <td>6.584415e-01</td>\n",
       "      <td>0.098940</td>\n",
       "      <td>0.131406</td>\n",
       "      <td>0.172212</td>\n",
       "      <td>0.192703</td>\n",
       "      <td>-95.982302</td>\n",
       "      <td>0.641147</td>\n",
       "      <td>0.300434</td>\n",
       "      <td>0.204178</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2250.484065</td>\n",
       "      <td>1.039785</td>\n",
       "      <td>0.116451</td>\n",
       "      <td>0.059952</td>\n",
       "      <td>0.107367</td>\n",
       "      <td>530.210725</td>\n",
       "      <td>0.030637</td>\n",
       "      <td>0.027440</td>\n",
       "      <td>0.034788</td>\n",
       "      <td>0.038875</td>\n",
       "      <td>0.057992</td>\n",
       "      <td>0.153455</td>\n",
       "      <td>1.726001e+06</td>\n",
       "      <td>2579.212724</td>\n",
       "      <td>1.434706e+07</td>\n",
       "      <td>5.128829e+07</td>\n",
       "      <td>1.940359e+07</td>\n",
       "      <td>1.940354e+07</td>\n",
       "      <td>72.019768</td>\n",
       "      <td>0.483519</td>\n",
       "      <td>0.440914</td>\n",
       "      <td>0.470582</td>\n",
       "      <td>1.980665</td>\n",
       "      <td>2.929242</td>\n",
       "      <td>0.486768</td>\n",
       "      <td>0.242592</td>\n",
       "      <td>2.929801</td>\n",
       "      <td>0.442972</td>\n",
       "      <td>1.712580</td>\n",
       "      <td>2.457799</td>\n",
       "      <td>0.063217</td>\n",
       "      <td>0.674613</td>\n",
       "      <td>0.271404</td>\n",
       "      <td>3.390149</td>\n",
       "      <td>0.832699</td>\n",
       "      <td>1.076276</td>\n",
       "      <td>689.302924</td>\n",
       "      <td>2.075599e+06</td>\n",
       "      <td>0.157373</td>\n",
       "      <td>164.898869</td>\n",
       "      <td>11.998292</td>\n",
       "      <td>0.150319</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.481125</td>\n",
       "      <td>0.075443</td>\n",
       "      <td>71.665655</td>\n",
       "      <td>6.581897</td>\n",
       "      <td>1.184489</td>\n",
       "      <td>0.470196</td>\n",
       "      <td>0.208155</td>\n",
       "      <td>124.342503</td>\n",
       "      <td>0.229291</td>\n",
       "      <td>0.405455</td>\n",
       "      <td>0.885692</td>\n",
       "      <td>1.619202</td>\n",
       "      <td>953.900687</td>\n",
       "      <td>4601.435964</td>\n",
       "      <td>0.315281</td>\n",
       "      <td>0.273331</td>\n",
       "      <td>0.451110</td>\n",
       "      <td>0.120751</td>\n",
       "      <td>1.830483e-01</td>\n",
       "      <td>0.292877</td>\n",
       "      <td>0.326057</td>\n",
       "      <td>0.213177</td>\n",
       "      <td>0.335292</td>\n",
       "      <td>0.116371</td>\n",
       "      <td>1.734275e-01</td>\n",
       "      <td>0.249505</td>\n",
       "      <td>0.300351</td>\n",
       "      <td>0.216997</td>\n",
       "      <td>0.340232</td>\n",
       "      <td>332.827628</td>\n",
       "      <td>0.479664</td>\n",
       "      <td>0.458447</td>\n",
       "      <td>0.403100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>8.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.028057</td>\n",
       "      <td>0.007801</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.170871</td>\n",
       "      <td>0.403309</td>\n",
       "      <td>0.005075</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.083167</td>\n",
       "      <td>0.001974</td>\n",
       "      <td>9.950000e+03</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000e+00</td>\n",
       "      <td>5.000000e+00</td>\n",
       "      <td>1.095421e+09</td>\n",
       "      <td>1.095421e+09</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-9.928014e+06</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-11.332080</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.355694</td>\n",
       "      <td>6.500000e-07</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000039</td>\n",
       "      <td>0.156027</td>\n",
       "      <td>8.890000e-07</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>-999.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1952.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.110542</td>\n",
       "      <td>0.068760</td>\n",
       "      <td>0.294989</td>\n",
       "      <td>478.000000</td>\n",
       "      <td>0.209035</td>\n",
       "      <td>0.642060</td>\n",
       "      <td>0.076385</td>\n",
       "      <td>0.107278</td>\n",
       "      <td>0.131467</td>\n",
       "      <td>0.060724</td>\n",
       "      <td>7.221708e+05</td>\n",
       "      <td>721.000000</td>\n",
       "      <td>7.230000e+02</td>\n",
       "      <td>2.213000e+03</td>\n",
       "      <td>1.103136e+09</td>\n",
       "      <td>1.103136e+09</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>67.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>107.000000</td>\n",
       "      <td>189.736571</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.355694</td>\n",
       "      <td>3.743169e-01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.090909</td>\n",
       "      <td>0.047821</td>\n",
       "      <td>0.156027</td>\n",
       "      <td>5.117519e-01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.048295</td>\n",
       "      <td>0.001483</td>\n",
       "      <td>14.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3766.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.159285</td>\n",
       "      <td>0.094513</td>\n",
       "      <td>0.345575</td>\n",
       "      <td>754.000000</td>\n",
       "      <td>0.230394</td>\n",
       "      <td>0.660669</td>\n",
       "      <td>0.096357</td>\n",
       "      <td>0.127504</td>\n",
       "      <td>0.159598</td>\n",
       "      <td>0.156245</td>\n",
       "      <td>9.578745e+05</td>\n",
       "      <td>1116.000000</td>\n",
       "      <td>2.040501e+07</td>\n",
       "      <td>1.040504e+08</td>\n",
       "      <td>1.112980e+09</td>\n",
       "      <td>1.112980e+09</td>\n",
       "      <td>11.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.166667</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>140.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>151.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.176471</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>113.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>275.000000</td>\n",
       "      <td>840.000006</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.355694</td>\n",
       "      <td>5.676439e-01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.186970</td>\n",
       "      <td>0.156027</td>\n",
       "      <td>7.115475e-01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.122595</td>\n",
       "      <td>0.005797</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>5781.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.247704</td>\n",
       "      <td>0.137316</td>\n",
       "      <td>0.428822</td>\n",
       "      <td>1151.000000</td>\n",
       "      <td>0.252082</td>\n",
       "      <td>0.676588</td>\n",
       "      <td>0.119282</td>\n",
       "      <td>0.150582</td>\n",
       "      <td>0.198533</td>\n",
       "      <td>0.298914</td>\n",
       "      <td>2.722813e+06</td>\n",
       "      <td>1419.000000</td>\n",
       "      <td>2.040510e+07</td>\n",
       "      <td>1.040511e+08</td>\n",
       "      <td>1.138371e+09</td>\n",
       "      <td>1.138371e+09</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>396.000000</td>\n",
       "      <td>8.000000e+00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>276.000000</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1.656250</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>210.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>669.000001</td>\n",
       "      <td>2404.499998</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.597865</td>\n",
       "      <td>6.591692e-01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.091463</td>\n",
       "      <td>0.230769</td>\n",
       "      <td>0.614582</td>\n",
       "      <td>0.376427</td>\n",
       "      <td>7.726099e-01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.009561</td>\n",
       "      <td>0.122595</td>\n",
       "      <td>0.259648</td>\n",
       "      <td>34.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>7783.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>0.752498</td>\n",
       "      <td>0.430576</td>\n",
       "      <td>0.932990</td>\n",
       "      <td>3057.000000</td>\n",
       "      <td>0.440870</td>\n",
       "      <td>0.723990</td>\n",
       "      <td>0.402483</td>\n",
       "      <td>0.543463</td>\n",
       "      <td>0.837402</td>\n",
       "      <td>0.709200</td>\n",
       "      <td>6.355811e+06</td>\n",
       "      <td>22761.000000</td>\n",
       "      <td>1.000000e+09</td>\n",
       "      <td>1.040515e+08</td>\n",
       "      <td>1.180218e+09</td>\n",
       "      <td>1.180218e+09</td>\n",
       "      <td>9999.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>91.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>73.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>11663.000000</td>\n",
       "      <td>4.840000e+07</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1378.000000</td>\n",
       "      <td>221.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>9999.000000</td>\n",
       "      <td>105.000000</td>\n",
       "      <td>37.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>962.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>56.000000</td>\n",
       "      <td>92.709045</td>\n",
       "      <td>12459.000000</td>\n",
       "      <td>310590.000100</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.680982</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.999676</td>\n",
       "      <td>0.505313</td>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.999377</td>\n",
       "      <td>54.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           studentId  MiddleSchoolId        AveKnow  AveCarelessness  \\\n",
       "count  942816.000000   942816.000000  942816.000000    942816.000000   \n",
       "mean     3844.844105        2.515472       0.195155         0.109436   \n",
       "std      2250.484065        1.039785       0.116451         0.059952   \n",
       "min         8.000000        1.000000       0.028057         0.007801   \n",
       "25%      1952.000000        2.000000       0.110542         0.068760   \n",
       "50%      3766.000000        2.000000       0.159285         0.094513   \n",
       "75%      5781.000000        4.000000       0.247704         0.137316   \n",
       "max      7783.000000        4.000000       0.752498         0.430576   \n",
       "\n",
       "          AveCorrect     NumActions    AveResBored   AveResEngcon  \\\n",
       "count  942816.000000  942816.000000  942816.000000  942816.000000   \n",
       "mean        0.372681     869.850594       0.232949       0.658442   \n",
       "std         0.107367     530.210725       0.030637       0.027440   \n",
       "min         0.000000       2.000000       0.170871       0.403309   \n",
       "25%         0.294989     478.000000       0.209035       0.642060   \n",
       "50%         0.345575     754.000000       0.230394       0.660669   \n",
       "75%         0.428822    1151.000000       0.252082       0.676588   \n",
       "max         0.932990    3057.000000       0.440870       0.723990   \n",
       "\n",
       "          AveResConf    AveResFrust  AveResOfftask   AveResGaming  \\\n",
       "count  942816.000000  942816.000000  942816.000000  942816.000000   \n",
       "mean        0.098940       0.131406       0.172212       0.192703   \n",
       "std         0.034788       0.038875       0.057992       0.153455   \n",
       "min         0.005075       0.000000       0.083167       0.001974   \n",
       "25%         0.076385       0.107278       0.131467       0.060724   \n",
       "50%         0.096357       0.127504       0.159598       0.156245   \n",
       "75%         0.119282       0.150582       0.198533       0.298914   \n",
       "max         0.402483       0.543463       0.837402       0.709200   \n",
       "\n",
       "         action_num      problemId  assignmentId  assistmentId     startTime  \\\n",
       "count  9.428160e+05  942816.000000  9.428160e+05  9.428160e+05  9.428160e+05   \n",
       "mean   1.849329e+06    1899.719319  1.198773e+07  6.061572e+07  1.120793e+09   \n",
       "std    1.726001e+06    2579.212724  1.434706e+07  5.128829e+07  1.940359e+07   \n",
       "min    9.950000e+03       1.000000  2.000000e+00  5.000000e+00  1.095421e+09   \n",
       "25%    7.221708e+05     721.000000  7.230000e+02  2.213000e+03  1.103136e+09   \n",
       "50%    9.578745e+05    1116.000000  2.040501e+07  1.040504e+08  1.112980e+09   \n",
       "75%    2.722813e+06    1419.000000  2.040510e+07  1.040511e+08  1.138371e+09   \n",
       "max    6.355811e+06   22761.000000  1.000000e+09  1.040515e+08  1.180218e+09   \n",
       "\n",
       "            endTime      timeTaken        correct       original  \\\n",
       "count  9.428160e+05  942816.000000  942816.000000  942816.000000   \n",
       "mean   1.120793e+09      29.747869       0.372681       0.264214   \n",
       "std    1.940354e+07      72.019768       0.483519       0.440914   \n",
       "min    1.095421e+09       0.000000       0.000000       0.000000   \n",
       "25%    1.103136e+09       5.000000       0.000000       0.000000   \n",
       "50%    1.112980e+09      11.000000       0.000000       0.000000   \n",
       "75%    1.138371e+09      30.000000       1.000000       1.000000   \n",
       "max    1.180218e+09    9999.000000       1.000000       1.000000   \n",
       "\n",
       "                hint      hintCount      hintTotal       scaffold  \\\n",
       "count  942816.000000  942816.000000  942816.000000  942816.000000   \n",
       "mean        0.331025       1.218490       1.953967       0.385732   \n",
       "std         0.470582       1.980665       2.929242       0.486768   \n",
       "min         0.000000       0.000000       0.000000       0.000000   \n",
       "25%         0.000000       0.000000       0.000000       0.000000   \n",
       "50%         0.000000       0.000000       1.000000       0.000000   \n",
       "75%         1.000000       2.000000       3.000000       1.000000   \n",
       "max         1.000000      56.000000      56.000000       1.000000   \n",
       "\n",
       "          bottomHint   attemptCount  frIsHelpRequest  frPast5HelpRequest  \\\n",
       "count  942816.000000  942816.000000    942816.000000       942816.000000   \n",
       "mean        0.062794       2.673605         0.268104            1.947322   \n",
       "std         0.242592       2.929801         0.442972            1.712580   \n",
       "min         0.000000       1.000000         0.000000            0.000000   \n",
       "25%         0.000000       1.000000         0.000000            0.000000   \n",
       "50%         0.000000       2.000000         0.000000            2.000000   \n",
       "75%         0.000000       3.000000         1.000000            3.000000   \n",
       "max         1.000000      91.000000         1.000000            5.000000   \n",
       "\n",
       "       frPast8HelpRequest    stlHintUsed  past8BottomOut  \\\n",
       "count       942816.000000  942816.000000   942816.000000   \n",
       "mean             2.575323       0.004012        0.241678   \n",
       "std              2.457799       0.063217        0.674613   \n",
       "min              0.000000       0.000000        0.000000   \n",
       "25%              0.000000       0.000000        0.000000   \n",
       "50%              2.000000       0.000000        0.000000   \n",
       "75%              4.000000       0.000000        0.000000   \n",
       "max              8.000000       1.000000        8.000000   \n",
       "\n",
       "       totalFrPercentPastWrong  totalFrPastWrongCount  frPast5WrongCount  \\\n",
       "count            942816.000000          942816.000000      942816.000000   \n",
       "mean                  0.227882               1.988008           0.719380   \n",
       "std                   0.271404               3.390149           0.832699   \n",
       "min                   0.000000               0.000000           0.000000   \n",
       "25%                   0.000000               0.000000           0.000000   \n",
       "50%                   0.166667               1.000000           1.000000   \n",
       "75%                   0.333333               2.000000           1.000000   \n",
       "max                   1.000000              73.000000           5.000000   \n",
       "\n",
       "       frPast8WrongCount  totalFrTimeOnSkill  timeSinceSkill  \\\n",
       "count      942816.000000       942816.000000    9.428160e+05   \n",
       "mean            0.944750          376.213405    4.850802e+05   \n",
       "std             1.076276          689.302924    2.075599e+06   \n",
       "min             0.000000            0.000000   -9.928014e+06   \n",
       "25%             0.000000           37.000000    0.000000e+00   \n",
       "50%             1.000000          140.000000    0.000000e+00   \n",
       "75%             1.000000          396.000000    8.000000e+00   \n",
       "max             8.000000        11663.000000    4.840000e+07   \n",
       "\n",
       "       frWorkingInSchool  totalFrAttempted  totalFrSkillOpportunities  \\\n",
       "count      942816.000000     942816.000000              942816.000000   \n",
       "mean            0.974588        193.316005                   8.381019   \n",
       "std             0.157373        164.898869                  11.998292   \n",
       "min             0.000000          0.000000                   0.000000   \n",
       "25%             1.000000         67.000000                   2.000000   \n",
       "50%             1.000000        151.000000                   4.000000   \n",
       "75%             1.000000        276.000000                  10.000000   \n",
       "max             1.000000       1378.000000                 221.000000   \n",
       "\n",
       "       responseIsFillIn  responseIsChosen  endsWithScaffolding  \\\n",
       "count     942816.000000          942816.0        942816.000000   \n",
       "mean           0.023131               0.0             0.636085   \n",
       "std            0.150319               0.0             0.481125   \n",
       "min            0.000000               0.0             0.000000   \n",
       "25%            0.000000               0.0             0.000000   \n",
       "50%            0.000000               0.0             1.000000   \n",
       "75%            0.000000               0.0             1.000000   \n",
       "max            1.000000               0.0             1.000000   \n",
       "\n",
       "       endsWithAutoScaffolding  frTimeTakenOnScaffolding  \\\n",
       "count            942816.000000             942816.000000   \n",
       "mean                  0.005724                 24.060853   \n",
       "std                   0.075443                 71.665655   \n",
       "min                   0.000000                  0.000000   \n",
       "25%                   0.000000                  0.000000   \n",
       "50%                   0.000000                  8.000000   \n",
       "75%                   0.000000                 23.000000   \n",
       "max                   1.000000               9999.000000   \n",
       "\n",
       "       frTotalSkillOpportunitiesScaffolding  \\\n",
       "count                         942816.000000   \n",
       "mean                               3.989371   \n",
       "std                                6.581897   \n",
       "min                                0.000000   \n",
       "25%                                0.000000   \n",
       "50%                                2.000000   \n",
       "75%                                5.000000   \n",
       "max                              105.000000   \n",
       "\n",
       "       totalFrSkillOpportunitiesByScaffolding  frIsHelpRequestScaffolding  \\\n",
       "count                           942816.000000               942816.000000   \n",
       "mean                                 1.036189                    0.670047   \n",
       "std                                  1.184489                    0.470196   \n",
       "min                                  0.000000                    0.000000   \n",
       "25%                                  0.000000                    0.000000   \n",
       "50%                                  1.176471                    1.000000   \n",
       "75%                                  1.656250                    1.000000   \n",
       "max                                 37.000000                    1.000000   \n",
       "\n",
       "       timeGreater5Secprev2wrong       sumRight  helpAccessUnder2Sec  \\\n",
       "count              942816.000000  942816.000000        942816.000000   \n",
       "mean                    0.045388     145.982059             0.055674   \n",
       "std                     0.208155     124.342503             0.229291   \n",
       "min                     0.000000       0.000000             0.000000   \n",
       "25%                     0.000000      50.000000             0.000000   \n",
       "50%                     0.000000     113.000000             0.000000   \n",
       "75%                     0.000000     210.000000             0.000000   \n",
       "max                     1.000000     962.000000             1.000000   \n",
       "\n",
       "       timeGreater10SecAndNextActionRight  consecutiveErrorsInRow  \\\n",
       "count                       942816.000000           942816.000000   \n",
       "mean                             0.207414                0.155307   \n",
       "std                              0.405455                0.885692   \n",
       "min                              0.000000                0.000000   \n",
       "25%                              0.000000                0.000000   \n",
       "50%                              0.000000                0.000000   \n",
       "75%                              0.000000                0.000000   \n",
       "max                              1.000000               56.000000   \n",
       "\n",
       "       sumTime3SDWhen3RowRight  sumTimePerSkill  \\\n",
       "count            942731.000000    942816.000000   \n",
       "mean                  0.087042       601.665586   \n",
       "std                   1.619202       953.900687   \n",
       "min                 -11.332080         0.000000   \n",
       "25%                   0.000000       107.000000   \n",
       "50%                   0.000000       275.000000   \n",
       "75%                   0.000000       669.000001   \n",
       "max                  92.709045     12459.000000   \n",
       "\n",
       "       totalTimeByPercentCorrectForskill     Prev5count     timeOver80  \\\n",
       "count                      942816.000000  942816.000000  942816.000000   \n",
       "mean                         2166.143744       4.972689       0.081323   \n",
       "std                          4601.435964       0.315281       0.273331   \n",
       "min                             0.000000       0.000000       0.000000   \n",
       "25%                           189.736571       5.000000       0.000000   \n",
       "50%                           840.000006       5.000000       0.000000   \n",
       "75%                          2404.499998       5.000000       0.000000   \n",
       "max                        310590.000100       5.000000       1.000000   \n",
       "\n",
       "           manywrong  confidence(BORED)  confidence(CONCENTRATING)  \\\n",
       "count  942816.000000      942816.000000               9.428160e+05   \n",
       "mean        0.715638           0.436958               5.400894e-01   \n",
       "std         0.451110           0.120751               1.830483e-01   \n",
       "min         0.000000           0.355694               6.500000e-07   \n",
       "25%         0.000000           0.355694               3.743169e-01   \n",
       "50%         1.000000           0.355694               5.676439e-01   \n",
       "75%         1.000000           0.597865               6.591692e-01   \n",
       "max         1.000000           0.680982               1.000000e+00   \n",
       "\n",
       "       confidence(CONFUSED)  confidence(FRUSTRATED)  confidence(OFF TASK)  \\\n",
       "count         942816.000000           942816.000000         942816.000000   \n",
       "mean               0.134450                0.164114              0.256006   \n",
       "std                0.292877                0.326057              0.213177   \n",
       "min                0.000000                0.000000              0.000000   \n",
       "25%                0.000000                0.000000              0.090909   \n",
       "50%                0.000000                0.000000              0.230769   \n",
       "75%                0.000000                0.091463              0.230769   \n",
       "max                1.000000                1.000000              1.000000   \n",
       "\n",
       "       confidence(GAMING)      RES_BORED  RES_CONCENTRATING   RES_CONFUSED  \\\n",
       "count       942816.000000  942816.000000       9.428160e+05  942816.000000   \n",
       "mean             0.337888       0.232949       6.584415e-01       0.098940   \n",
       "std              0.335292       0.116371       1.734275e-01       0.249505   \n",
       "min              0.000039       0.156027       8.890000e-07       0.000000   \n",
       "25%              0.047821       0.156027       5.117519e-01       0.000000   \n",
       "50%              0.186970       0.156027       7.115475e-01       0.000000   \n",
       "75%              0.614582       0.376427       7.726099e-01       0.000000   \n",
       "max              0.999676       0.505313       1.000000e+00       1.000000   \n",
       "\n",
       "       RES_FRUSTRATED    RES_OFFTASK     RES_GAMING           MCAS  \\\n",
       "count   942816.000000  942816.000000  942816.000000  942816.000000   \n",
       "mean         0.131406       0.172212       0.192703     -95.982302   \n",
       "std          0.300351       0.216997       0.340232     332.827628   \n",
       "min          0.000000       0.000000       0.000001    -999.000000   \n",
       "25%          0.000000       0.048295       0.001483      14.000000   \n",
       "50%          0.000000       0.122595       0.005797      23.000000   \n",
       "75%          0.009561       0.122595       0.259648      34.000000   \n",
       "max          1.000000       1.000000       0.999377      54.000000   \n",
       "\n",
       "            Enrolled      Selective         isSTEM  \n",
       "count  942816.000000  942816.000000  316974.000000  \n",
       "mean        0.641147       0.300434       0.204178  \n",
       "std         0.479664       0.458447       0.403100  \n",
       "min         0.000000       0.000000       0.000000  \n",
       "25%         0.000000       0.000000       0.000000  \n",
       "50%         1.000000       0.000000       0.000000  \n",
       "75%         1.000000       1.000000       0.000000  \n",
       "max         1.000000       1.000000       1.000000  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "9e422eeb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number of records: 942816\n"
     ]
    }
   ],
   "source": [
    "print(\"The number of records: \" + str(len(data['action_num'].unique())))\n",
    "#or use print(data['action_num'].count())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "e3282f0f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Part of missing values for every column\n",
      "studentId               0.000000\n",
      "MiddleSchoolId          0.000000\n",
      "InferredGender          0.184189\n",
      "SY ASSISTments Usage    0.000000\n",
      "AveKnow                 0.000000\n",
      "                          ...   \n",
      "Ln                      0.000000\n",
      "MCAS                    0.000000\n",
      "Enrolled                0.000000\n",
      "Selective               0.000000\n",
      "isSTEM                  0.663801\n",
      "Length: 82, dtype: float64\n",
      "studentId               942816\n",
      "MiddleSchoolId          942816\n",
      "InferredGender          769160\n",
      "SY ASSISTments Usage    942816\n",
      "AveKnow                 942816\n",
      "                         ...  \n",
      "Ln                      942816\n",
      "MCAS                    942816\n",
      "Enrolled                942816\n",
      "Selective               942816\n",
      "isSTEM                  316974\n",
      "Length: 82, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print('Part of missing values for every column')\n",
    "print(data.isnull().sum() / len(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "46d6be28",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1709"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(data.studentId.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "09fae4e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(data.MiddleSchoolId.unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db43bd95",
   "metadata": {},
   "source": [
    "*****\n",
    "## Sort by student id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "5f6638cd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-3e81dc\"><g class=\"clips\"><clipPath id=\"clip3e81dcxyplot\" class=\"plotclip\"><rect width=\"540\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip3e81dcx\"><rect x=\"80\" y=\"0\" width=\"540\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip3e81dcy\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip3e81dcxy\"><rect x=\"80\" y=\"100\" width=\"540\" height=\"320\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"540\" height=\"320\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(163.91,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(247.81,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(331.72,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(415.62,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(499.53,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(583.4300000000001,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"/></g><g class=\"zerolinelayer\"><path class=\"xzl zl crisp\" transform=\"translate(80,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\" clip-path=\"url(#clip3e81dcxyplot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M0,319.2V312.8H286.29V319.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,311.2V304.8H289.31V311.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,303.2V296.8H289.47V303.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,295.2V288.8H290.65V295.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,287.2V280.8H292.83V287.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,279.2V272.8H294.34V279.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,271.2V264.8H295.18V271.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,263.2V256.8H296.19V263.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,255.2V248.8H296.36V255.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,247.2V240.8H297.19V247.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,239.2V232.8H302.73V239.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,231.2V224.8H304.07V231.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,223.2V216.8H304.41V223.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,215.2V208.8H310.62V215.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,207.2V200.8H311.29V207.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,199.2V192.8H311.63V199.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,191.2V184.8H312.63V191.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,183.2V176.8H314.48V183.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,175.2V168.8H315.65V175.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,167.2V160.8H323.04V167.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,159.2V152.8H323.88V159.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,151.2V144.8H325.89V151.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,143.2V136.8H331.6V143.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,135.2V128.8H339.65V135.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,127.2V120.8H343.01V127.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,119.2V112.8H344.52V119.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,111.2V104.8H344.68V111.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,103.2V96.8H351.73V103.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,95.2V88.8H353.58V95.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,87.2V80.8H354.42V87.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,79.2V72.8H371.7V79.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,71.2V64.8H372.37V71.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,63.2V56.8H379.25V63.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,55.2V48.8H381.77V55.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,47.2V40.8H391.84V47.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,39.2V32.8H398.55V39.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,31.2V24.8H418.86V31.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,23.2V16.8H435.14V23.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,15.2V8.8H460.14V15.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,7.2V0.8H513V7.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(80,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(163.91,0)\">500</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(247.81,0)\">1000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(331.72,0)\">1500</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(415.62,0)\">2000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(499.53,0)\">2500</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(583.4300000000001,0)\">3000</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,416)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">118-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,400)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">6694-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,384)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">3432-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,368)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">6267-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,352)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">832-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,336)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">5599-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,320)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">386-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,304)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">3778-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,288)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1764-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,272)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1450-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,256)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">6862-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,240)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">2642-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,224)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">7175-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,208)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">6036-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,192)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">4654-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,176)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">2323-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,160)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">5130-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,144)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">3069-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,128)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">954-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,112)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">3609-</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-3e81dc\"><g class=\"clips\"/></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Top 40 students by number of actions</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"350\" y=\"460.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">count</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,18.231250000000003,260)\" x=\"18.231250000000003\" y=\"260\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">studentId</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = data['studentId'].value_counts().reset_index() #value_counts以后studentid是index，需要reset \n",
    "\n",
    "ds.columns = [\n",
    "    'studentId',\n",
    "    'count'\n",
    "]\n",
    "\n",
    "ds['studentId'] = ds['studentId'].astype(str) + '-' #将数据转成str类。否则纵坐标出错\n",
    "ds = ds.sort_values(['count']).tail(40)\n",
    "\n",
    "fig = px.bar(\n",
    "    ds,\n",
    "    x = 'count',\n",
    "    y = 'studentId',\n",
    "    orientation='h',\n",
    "    title='Top 40 students by number of actions'\n",
    ")\n",
    "\n",
    "fig.show(\"svg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "467e483e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-1b6f13\"><g class=\"clips\"><clipPath id=\"clip1b6f13xyplot\" class=\"plotclip\"><rect width=\"540\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip1b6f13x\"><rect x=\"80\" y=\"0\" width=\"540\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip1b6f13y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip1b6f13xy\"><rect x=\"80\" y=\"100\" width=\"540\" height=\"320\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"540\" height=\"320\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"/><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,377.84)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,335.66999999999996)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,293.51)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,251.35)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,209.18)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,167.01999999999998)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,124.85)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"yzl zl crisp\" transform=\"translate(0,420)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\" clip-path=\"url(#clip1b6f13xyplot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" shape-rendering=\"crispEdges\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M0,320V66.47H33.75V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M33.75,320V16H67.5V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M67.5,320V91.83H101.25V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M101.25,320V76.65H135V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M135,320V43.48H168.75V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M168.75,320V33.57H202.5V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M202.5,320V56.81H236.25V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M236.25,320V56.33H270V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M270,320V103H303.75V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M303.75,320V54.49H337.5V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M337.5,320V99.92H371.25V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M371.25,320V77.32H405V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M405,320V51.77H438.75V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M438.75,320V79.11H472.5V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M472.5,320V69.79H506.25V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M506.25,320V168.19H540V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(80.03,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(215.03,0)\">2000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(350.03,0)\">4000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(485.03,0)\">6000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(620.03,0)\">8000</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,420)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,377.84)\">10k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,335.66999999999996)\">20k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,293.51)\">30k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,251.35)\">40k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,209.18)\">50k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,167.01999999999998)\">60k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,124.85)\">70k</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-1b6f13\"><g class=\"clips\"/></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">User action distribution</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"350\" y=\"460.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">studentId</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,31.840625000000003,260)\" x=\"31.840625000000003\" y=\"260\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">sum of count</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = data['studentId'].value_counts().reset_index()\n",
    "\n",
    "ds.columns = [\n",
    "    'studentId',\n",
    "    'count'\n",
    "]\n",
    "## Correct answers\n",
    "ds = ds.sort_values('studentId')\n",
    "\n",
    "fig = px.histogram(\n",
    "    ds,\n",
    "    x = 'studentId',\n",
    "    y = 'count',\n",
    "    title = 'User action distribution'\n",
    ")\n",
    "\n",
    "fig.show(\"svg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17a0367d",
   "metadata": {},
   "source": [
    "****\n",
    "## Sort by MiddleSchoolId"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "c7c54c5e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-79b24a\"><g class=\"clips\"/><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"/><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"><g class=\"trace\" stroke-linejoin=\"round\" style=\"opacity: 1;\"><g class=\"slice\"><path class=\"surface\" d=\"M350,260l0,-160a160,160 0 1 1 -11.158128387110828,319.61045132101077Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(429.951291586942,267.54123155252023)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\">51.1%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M350,260l-155.92654148371372,35.87357886979919a160,160 0 0 1 155.92654148371372,-195.8735788697992Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(279.768049588679,208.841361415455)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">28.6%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M350,260l-80.84436518714652,138.07312778918023a160,160 0 0 1 -75.0821762965672,-102.19954891938104Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(0, 204, 150); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(257.65360800066867,332.59343138260977)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">13%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M350,260l-11.158128387110867,159.61045132101077a160,160 0 0 1 -69.68623680003566,-21.537323531830538Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(171, 99, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(311.5248668643327,389.2402707916524)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">7.32%</text></g></g></g></g><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-79b24a\"><g class=\"clips\"/><clipPath id=\"legend79b24a\"><rect width=\"53\" height=\"86\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(630.8,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"53\" height=\"86\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend79b24a)\"><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,14.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">2</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"47.640625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,33.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">4</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"47.640625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,52.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(0, 204, 150); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"47.640625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,71.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">3</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(171, 99, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"47.640625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Percent of schools</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = data['MiddleSchoolId'].value_counts().reset_index()\n",
    "\n",
    "ds.columns = [\n",
    "    'MiddleSchoolId',\n",
    "    'percent'\n",
    "]\n",
    "\n",
    "ds['percent'] /= len(data)\n",
    "ds = ds.sort_values(['percent'])\n",
    "\n",
    "fig = px.pie(\n",
    "    ds,\n",
    "    names = 'MiddleSchoolId',\n",
    "    values = 'percent',\n",
    "    title = 'Percent of schools',\n",
    ")\n",
    "\n",
    "fig.show(\"svg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e53a3ba5",
   "metadata": {},
   "source": [
    "*****\n",
    "## Sort by correct answers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "4c1d3a9e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-310b35\"><g class=\"clips\"/><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"/><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"><g class=\"trace\" stroke-linejoin=\"round\" style=\"opacity: 1;\"><g class=\"slice\"><path class=\"surface\" d=\"M350,260l0,-160a160,160 0 1 1 -114.77321438390072,271.47694497064026Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(423.6854049280643,295.90222464908715)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\">62.7%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M350,260l-114.77321438390071,111.4769449706403a160,160 0 0 1 114.77321438390071,-271.4769449706403Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(273.2870239434337,232.31779899700314)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">37.3%</text></g></g></g></g><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-310b35\"><g class=\"clips\"/><clipPath id=\"legend310b35\"><rect width=\"53\" height=\"48\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(630.8,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"53\" height=\"48\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend310b35)\"><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,14.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"47.640625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,33.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"47.640625\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Percent of correct answers</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = data['correct'].value_counts().reset_index()\n",
    "\n",
    "ds.columns = [\n",
    "    'correct',\n",
    "    'percent'\n",
    "]\n",
    "\n",
    "ds['percent'] /= len(data)\n",
    "ds = ds.sort_values(['percent'])\n",
    "\n",
    "fig = px.pie(\n",
    "    ds,\n",
    "    names = ['0', '1'],\n",
    "    values = 'percent',\n",
    "    title = 'Percent of correct answers'    \n",
    ")\n",
    "\n",
    "fig.show(\"svg\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "800e7096",
   "metadata": {},
   "source": [
    "*******\n",
    "## Sort by problem id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "b1b6089c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-59fdf3\"><g class=\"clips\"><clipPath id=\"clip59fdf3xyplot\" class=\"plotclip\"><rect width=\"540\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip59fdf3x\"><rect x=\"80\" y=\"0\" width=\"540\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip59fdf3y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip59fdf3xy\"><rect x=\"80\" y=\"100\" width=\"540\" height=\"320\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"540\" height=\"320\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(183.68,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(287.36,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(391.03,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(494.71,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(598.39,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"/></g><g class=\"zerolinelayer\"><path class=\"xzl zl crisp\" transform=\"translate(80,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\" clip-path=\"url(#clip59fdf3xyplot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M0,319.2V312.8H181.33V319.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,311.2V304.8H181.44V311.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,303.2V296.8H181.85V303.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,295.2V288.8H183.2V295.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,287.2V280.8H186V287.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,279.2V272.8H187.97V279.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,271.2V264.8H188.38V271.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,263.2V256.8H190.56V263.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,255.2V248.8H192.32V255.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,247.2V240.8H194.81V247.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,239.2V232.8H195.95V239.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,231.2V224.8H196.47V231.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,223.2V216.8H197.4V223.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,215.2V208.8H198.13V215.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,207.2V200.8H198.34V207.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,199.2V192.8H198.96V199.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,191.2V184.8H199.68V191.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,183.2V176.8H201.24V183.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,175.2V168.8H203.62V175.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,167.2V160.8H204.04V167.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,159.2V152.8H207.05V159.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,151.2V144.8H212.75V151.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,143.2V136.8H215.34V143.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,135.2V128.8H217.21V135.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,127.2V120.8H220.32V127.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,119.2V112.8H224.15V119.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,111.2V104.8H233.59V111.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,103.2V96.8H250.9V103.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,95.2V88.8H258.26V95.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,87.2V80.8H268.84V87.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,79.2V72.8H276.2V79.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,71.2V64.8H287.09V71.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,63.2V56.8H298.7V63.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,55.2V48.8H313.42V55.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,47.2V40.8H323.06V47.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,39.2V32.8H357.38V39.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,31.2V24.8H365.47V31.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,23.2V16.8H377.7V23.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,15.2V8.8H409.84V15.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,7.2V0.8H513V7.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(80,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(183.68,0)\">1000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(287.36,0)\">2000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(391.03,0)\">3000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(494.71,0)\">4000</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(598.39,0)\">5000</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,416)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">171-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,400)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1080-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,384)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1091-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,368)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1233-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,352)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1178-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,336)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">834-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,320)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1226-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,304)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">4665-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,288)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1268-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,272)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1079-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,256)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1179-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,240)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1261-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,224)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1096-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,208)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1216-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,192)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">804-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,176)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1138-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,160)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1181-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,144)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">196-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,128)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1202-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,112)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">1180-</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-59fdf3\"><g class=\"clips\"/></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Top 40 useful problem ids</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"350\" y=\"460.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">count</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,18.231250000000003,260)\" x=\"18.231250000000003\" y=\"260\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">problemId</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = data['problemId'].value_counts().reset_index()\n",
    "\n",
    "ds.columns = [\n",
    "    'problemId',\n",
    "    'count'\n",
    "]\n",
    "\n",
    "ds['problemId'] = ds['problemId'].astype(str) + '-'\n",
    "ds = ds.sort_values(['count']).tail(40)\n",
    "\n",
    "fig = px.bar(\n",
    "    ds,\n",
    "    x = 'count',\n",
    "    y = 'problemId',\n",
    "    orientation = 'h',\n",
    "    title = 'Top 40 useful problem ids'\n",
    ")\n",
    "\n",
    "fig.show(\"svg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "97befdaa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-7f4729\"><g class=\"clips\"><clipPath id=\"clip7f4729xyplot\" class=\"plotclip\"><rect width=\"540\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip7f4729x\"><rect x=\"80\" y=\"0\" width=\"540\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip7f4729y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clip7f4729xy\"><rect x=\"80\" y=\"100\" width=\"540\" height=\"320\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"540\" height=\"320\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"/><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,374.28)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,328.56)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,282.84000000000003)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,237.11)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,191.39)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,145.67000000000002)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"yzl zl crisp\" transform=\"translate(0,420)\" d=\"M80,0h540\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\" clip-path=\"url(#clip7f4729xyplot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" shape-rendering=\"crispEdges\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M0,320V162.45H11.74V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M11.74,320V122.82H23.48V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M23.48,320V16H35.22V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M35.22,320V303.22H46.96V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M46.96,320V277.93H58.7V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M58.7,320V304.86H70.43V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M70.43,320V304.68H82.17V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M82.17,320V315.52H93.91V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M93.91,320V307.71H105.65V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M105.65,320V285.97H117.39V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M187.83,320V319.57H199.57V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M199.57,320V314.48H211.3V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M211.3,320V296.09H223.04V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M223.04,320V304.42H234.78V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M234.78,320V314.18H246.52V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M246.52,320V312.59H258.26V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M270,320V319.99H281.74V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M281.74,320V319.44H293.48V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M293.48,320V319.56H305.22V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M305.22,320V319.63H316.96V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M316.96,320V318.63H328.7V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M328.7,320V319.97H340.43V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M340.43,320V319.63H352.17V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M352.17,320V319.96H363.91V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M363.91,320V319.94H375.65V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M481.3,320V319.99H493.04V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,0Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M504.78,320V319.85H516.52V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M516.52,320V318.93H528.26V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g><g class=\"point\"><path d=\"M528.26,320V319.88H540V320Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0px; fill: rgb(99, 110, 250); fill-opacity: 1;\"/></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(80.01,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(197.4,0)\">5k</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(314.78999999999996,0)\">10k</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(432.19,0)\">15k</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(549.5799999999999,0)\">20k</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,420)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,374.28)\">50k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,328.56)\">100k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,282.84000000000003)\">150k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,237.11)\">200k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,191.39)\">250k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,145.67000000000002)\">300k</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-7f4729\"><g class=\"clips\"/></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">problemid action distribution</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"350\" y=\"460.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">problemId</text></g><g class=\"g-ytitle\"><text class=\"ytitle\" transform=\"rotate(-90,24.200000000000003,260)\" x=\"24.200000000000003\" y=\"260\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">sum of count</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = data['problemId'].value_counts().reset_index()\n",
    "\n",
    "ds.columns = [\n",
    "    'problemId', \n",
    "    'count'\n",
    "]\n",
    "\n",
    "ds = ds.sort_values('problemId')\n",
    "\n",
    "fig = px.histogram(\n",
    "    ds, \n",
    "    x='problemId', \n",
    "    y='count', \n",
    "    title='problemid action distribution'\n",
    ")\n",
    "\n",
    "fig.show(\"svg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "79f85068",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-fc5047\"><g class=\"clips\"/><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"/><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"/><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"><g class=\"trace\" stroke-linejoin=\"round\" style=\"opacity: 1;\"><g class=\"slice\"><path class=\"surface\" d=\"M266,260l0,-160a160,160 0 0 1 157.0833494534803,190.41087510210434Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(335.6751701064983,207.27004205011505)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\">28%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l-154.48919392918023,-41.63038504640742a160,160 0 0 1 154.48919392918023,-118.36961495359259Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(205.49022471669412,185.7761314779242)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">20.8%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l-88.27944682769335,133.4418947249943a160,160 0 0 1 -66.20974710148688,-175.07227977140172Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(0, 204, 150); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(171.57533333623118,300.4600125051983)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">19.9%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l30.79811552855272,157.00788540672065a160,160 0 0 1 -119.07756235624606,-23.565990681726333Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(171, 99, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(243.48024776034225,378.5409811550777)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">12.4%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l102.78038486396242,122.62215332971348a160,160 0 0 1 -71.98226933540971,34.38573207700716Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(255, 161, 90); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(321.20456583638156,380.31391812975)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">8.02%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l132.5169230543504,89.6619490319465a160,160 0 0 1 -29.736538190387975,32.96020429776698Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(25, 211, 243); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(355.3664516727227,347.0234355164339)rotate(42.05662610732105)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">4.43%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l144.10501605366431,69.52513465052155a160,160 0 0 1 -11.588092999313915,20.136814381424955Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(255, 102, 146); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(382.2135722143181,332.3575587401092)rotate(29.919051013135118)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\">2.31%</text></g></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l149.88700889170073,55.981109005621086a160,160 0 0 1 -5.781992838036416,13.544025644900465Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(182, 232, 128); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(436.8706709542861,411.46112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">1.47%</text></g><path class=\"textline\" stroke-width=\"1.5\" d=\"M413.1519209542861,322.81967971474813V406.71112252928606h3.625\" fill=\"none\" style=\"stroke: rgb(42, 63, 95); stroke-opacity: 1;\"/></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l153.70123201527377,44.4514485364311a160,160 0 0 1 -3.8142231235730435,11.529660469189984Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(255, 151, 255); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(441.62229924056743,396.96112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">1.21%</text></g><path class=\"textline\" stroke-width=\"1.5\" d=\"M417.90354924056743,310.25247982058704V392.21112252928606h3.625\" fill=\"none\" style=\"stroke: rgb(42, 63, 95); stroke-opacity: 1;\"/></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l154.86364157738265,40.21507823431353a160,160 0 0 1 -1.162409562108877,4.236370302117571Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(254, 203, 82); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(447.82822667857,382.46112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0.437%</text></g><path class=\"textline\" stroke-width=\"1.5\" d=\"M420.29697667857,302.33725295591137V377.71112252928606h3.625\" fill=\"none\" style=\"stroke: rgb(42, 63, 95); stroke-opacity: 1;\"/></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l155.69679150548262,36.85795863715571a160,160 0 0 1 -0.8331499280999708,3.357119597157819Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(198, 202, 253); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(448.82053878757745,367.96112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0.344%</text></g><path class=\"textline\" stroke-width=\"1.5\" d=\"M421.28928878757745,298.5387699316985V363.21112252928606h3.625\" fill=\"none\" style=\"stroke: rgb(42, 63, 95); stroke-opacity: 1;\"/></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l156.3696206215514,33.89014232298365a160,160 0 0 1 -0.6728291160687832,2.9678163141720617Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(247, 167, 153); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(449.5715120411752,353.46112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0.303%</text></g><path class=\"textline\" stroke-width=\"1.5\" d=\"M422.0402620411752,295.37565013001705V348.71112252928606h3.625\" fill=\"none\" style=\"stroke: rgb(42, 63, 95); stroke-opacity: 1;\"/></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l156.70725685488102,32.292965937156374a160,160 0 0 1 -0.33763623332961856,1.5971763858272752Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(51, 255, 201); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(450.07172574253804,338.96112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0.162%</text></g><path class=\"textline\" stroke-width=\"1.5\" d=\"M422.54047574253804,293.0919847440414V334.21112252928606h3.625\" fill=\"none\" style=\"stroke: rgb(42, 63, 95); stroke-opacity: 1;\"/></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l156.96917572958702,30.994803931853273a160,160 0 0 1 -0.26191887470599795,1.2981620053031016Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(224, 198, 253); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(450.3708094098555,324.46112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0.132%</text></g><path class=\"textline\" stroke-width=\"1.5\" d=\"M422.8395594098555,291.6441559236838V319.71112252928606h3.625\" fill=\"none\" style=\"stroke: rgb(42, 63, 95); stroke-opacity: 1;\"/></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l157.06873938723783,30.486244558882078a160,160 0 0 1 -0.09956365765080477,0.5085593729711952Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(255, 219, 192); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(454.3629134511198,309.96112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0.0515%</text></g><path class=\"textline\" stroke-width=\"1.5\" d=\"M423.0191634511198,290.7405645541932V305.21112252928606h3.625\" fill=\"none\" style=\"stroke: rgb(42, 63, 95); stroke-opacity: 1;\"/></g><g class=\"slice\"><path class=\"surface\" d=\"M266,260l157.08334945348025,30.41087510210466a160,160 0 0 1 -0.014610066242426,0.07536945677741613Z\" style=\"pointer-events: all; stroke-width: 0; fill: rgb(122, 230, 248); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/><g class=\"slicetext\"><text data-notex=\"1\" class=\"slicetext\" transform=\"translate(458.2322989409043,295.46112252928606)\" text-anchor=\"middle\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0.00764%</text></g></g></g></g><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-fc5047\"><g class=\"clips\"/><clipPath id=\"legendfc5047\"><rect width=\"229\" height=\"314\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(459.44,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"229\" height=\"314\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legendfc5047)\"><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,14.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">textfieldquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,33.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">radioquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,52.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">noprobtype</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(0, 204, 150); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,71.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(171, 99, 250); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,90.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">algebrafieldquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(255, 161, 90); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,109.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">other</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(25, 211, 243); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,128.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">algebra</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(255, 102, 146); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,147.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">popupmenuquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(182, 232, 128); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,166.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">interfaceradioquestion1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(255, 151, 255); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,185.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">checkboxquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(254, 203, 82); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,204.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">interfacetextfieldquestion1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(198, 202, 253); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,223.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">algebrafieldquestion1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(247, 167, 153); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,242.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">interfaceradioquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(51, 255, 201); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,261.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">interfacetextfieldquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(224, 198, 253); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,280.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">interfacepopupmenuquestion1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(255, 219, 192); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,299.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">0</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendpie\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0; fill: rgb(122, 230, 248); fill-opacity: 1; stroke: rgb(68, 68, 68); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"223.03125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Percent of problem types</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = data['problemType'].value_counts().reset_index()\n",
    "\n",
    "ds.columns = [\n",
    "    'problemType',\n",
    "    'percent'\n",
    "]\n",
    "\n",
    "ds['percent'] /= len(data)\n",
    "ds = ds.sort_values(['percent'])\n",
    "\n",
    "fig = px.pie(\n",
    "    ds,\n",
    "    names = 'problemType',\n",
    "    values = 'percent',\n",
    "    title = 'Percent of problem types',\n",
    ")\n",
    "\n",
    "fig.show(\"svg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "0fa3cf54",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-82e46a\"><g class=\"clips\"><clipPath id=\"clip82e46axyplot\" class=\"plotclip\"><rect width=\"177.3\" height=\"85.33333333333334\"/></clipPath><clipPath id=\"clip82e46ax2y2plot\" class=\"plotclip\"><rect width=\"177.29999999999998\" height=\"85.33333333333334\"/></clipPath><clipPath id=\"clip82e46ax3y3plot\" class=\"plotclip\"><rect width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath id=\"clip82e46ax4y4plot\" class=\"plotclip\"><rect width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath id=\"clip82e46ax5y5plot\" class=\"plotclip\"><rect width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath id=\"clip82e46ax6y6plot\" class=\"plotclip\"><rect width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax\"><rect x=\"80\" y=\"0\" width=\"177.3\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ay\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46axy\"><rect x=\"80\" y=\"100\" width=\"177.3\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ay2\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46axy2\"><rect x=\"80\" y=\"100\" width=\"177.3\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ay3\"><rect x=\"0\" y=\"217.33333333333334\" width=\"700\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46axy3\"><rect x=\"80\" y=\"217.33333333333334\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ay4\"><rect x=\"0\" y=\"217.33333333333334\" width=\"700\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46axy4\"><rect x=\"80\" y=\"217.33333333333334\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ay5\"><rect x=\"0\" y=\"334.6666666666667\" width=\"700\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46axy5\"><rect x=\"80\" y=\"334.6666666666667\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ay6\"><rect x=\"0\" y=\"334.6666666666667\" width=\"700\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46axy6\"><rect x=\"80\" y=\"334.6666666666667\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax2\"><rect x=\"296.70000000000005\" y=\"0\" width=\"177.29999999999998\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax2y\"><rect x=\"296.70000000000005\" y=\"100\" width=\"177.29999999999998\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax2y2\"><rect x=\"296.70000000000005\" y=\"100\" width=\"177.29999999999998\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax2y3\"><rect x=\"296.70000000000005\" y=\"217.33333333333334\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax2y4\"><rect x=\"296.70000000000005\" y=\"217.33333333333334\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax2y5\"><rect x=\"296.70000000000005\" y=\"334.6666666666667\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax2y6\"><rect x=\"296.70000000000005\" y=\"334.6666666666667\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax3\"><rect x=\"80\" y=\"0\" width=\"177.3\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax3y\"><rect x=\"80\" y=\"100\" width=\"177.3\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax3y2\"><rect x=\"80\" y=\"100\" width=\"177.3\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax3y3\"><rect x=\"80\" y=\"217.33333333333334\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax3y4\"><rect x=\"80\" y=\"217.33333333333334\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax3y5\"><rect x=\"80\" y=\"334.6666666666667\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax3y6\"><rect x=\"80\" y=\"334.6666666666667\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax4\"><rect x=\"296.70000000000005\" y=\"0\" width=\"177.29999999999998\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax4y\"><rect x=\"296.70000000000005\" y=\"100\" width=\"177.29999999999998\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax4y2\"><rect x=\"296.70000000000005\" y=\"100\" width=\"177.29999999999998\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax4y3\"><rect x=\"296.70000000000005\" y=\"217.33333333333334\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax4y4\"><rect x=\"296.70000000000005\" y=\"217.33333333333334\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax4y5\"><rect x=\"296.70000000000005\" y=\"334.6666666666667\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax4y6\"><rect x=\"296.70000000000005\" y=\"334.6666666666667\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax5\"><rect x=\"80\" y=\"0\" width=\"177.3\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax5y\"><rect x=\"80\" y=\"100\" width=\"177.3\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax5y2\"><rect x=\"80\" y=\"100\" width=\"177.3\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax5y3\"><rect x=\"80\" y=\"217.33333333333334\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax5y4\"><rect x=\"80\" y=\"217.33333333333334\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax5y5\"><rect x=\"80\" y=\"334.6666666666667\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax5y6\"><rect x=\"80\" y=\"334.6666666666667\" width=\"177.3\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax6\"><rect x=\"296.70000000000005\" y=\"0\" width=\"177.29999999999998\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax6y\"><rect x=\"296.70000000000005\" y=\"100\" width=\"177.29999999999998\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax6y2\"><rect x=\"296.70000000000005\" y=\"100\" width=\"177.29999999999998\" height=\"85.33333333333334\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax6y3\"><rect x=\"296.70000000000005\" y=\"217.33333333333334\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax6y4\"><rect x=\"296.70000000000005\" y=\"217.33333333333334\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax6y5\"><rect x=\"296.70000000000005\" y=\"334.6666666666667\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath><clipPath class=\"axesclip\" id=\"clip82e46ax6y6\"><rect x=\"296.70000000000005\" y=\"334.6666666666667\" width=\"177.29999999999998\" height=\"85.33333333333333\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"80\" y=\"100\" width=\"177.3\" height=\"85.33333333333334\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/><rect class=\"bg\" x=\"296.70000000000005\" y=\"100\" width=\"177.29999999999998\" height=\"85.33333333333334\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/><rect class=\"bg\" x=\"80\" y=\"217.33333333333334\" width=\"177.3\" height=\"85.33333333333333\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/><rect class=\"bg\" x=\"296.70000000000005\" y=\"217.33333333333334\" width=\"177.29999999999998\" height=\"85.33333333333333\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/><rect class=\"bg\" x=\"80\" y=\"334.6666666666667\" width=\"177.3\" height=\"85.33333333333333\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/><rect class=\"bg\" x=\"296.70000000000005\" y=\"334.6666666666667\" width=\"177.29999999999998\" height=\"85.33333333333333\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"/><g class=\"y\"><path class=\"ygrid crisp\" transform=\"translate(0,151.7)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"ygrid crisp\" transform=\"translate(0,118.07)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"yzl zl crisp\" transform=\"translate(0,185.32999999999998)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,100)\" clip-path=\"url(#clip82e46axyplot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M8.87,85.33V4.27H79.79V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\" transform=\"translate(44.330000000000005,19.27)\">57.71%</text></g><g class=\"point\"><path d=\"M97.52,85.33V25.93H168.44V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(255, 255, 255); fill-opacity: 1; white-space: pre;\" transform=\"translate(132.98,40.93)\">42.29%</text></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"198.33333333333334\" transform=\"translate(124.33,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">wrong</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"198.33333333333334\" transform=\"translate(212.98,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">right</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,185.32999999999998)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,151.7)\">10k</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,118.07)\">20k</text></g></g><g class=\"overaxes-above\"/></g><g class=\"subplot x2y2\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x2\"/><g class=\"y2\"><path class=\"y2grid crisp\" transform=\"translate(0,154.26)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"y2grid crisp\" transform=\"translate(0,123.18)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"y2zl zl crisp\" transform=\"translate(0,185.32999999999998)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(296.70000000000005,100)\" clip-path=\"url(#clip82e46ax2y2plot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M8.86,85.33V4.27H79.79V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(44.325,19.27)\">69.0%</text></g><g class=\"point\"><path d=\"M97.51,85.33V48.92H168.43V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(132.97,63.92)\">31.0%</text></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"x2tick\"><text text-anchor=\"middle\" x=\"0\" y=\"198.33333333333334\" transform=\"translate(341.03000000000003,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">wrong</text></g><g class=\"x2tick\"><text text-anchor=\"middle\" x=\"0\" y=\"198.33333333333334\" transform=\"translate(429.68000000000006,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">right</text></g></g><g class=\"yaxislayer-above\"><g class=\"y2tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" transform=\"translate(0,185.32999999999998)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"y2tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,154.26)\">20k</text></g><g class=\"y2tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,123.18)\">40k</text></g></g><g class=\"overaxes-above\"/></g><g class=\"subplot x3y3\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x3\"/><g class=\"y3\"><path class=\"y3grid crisp\" transform=\"translate(0,278.58333333333337)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"y3grid crisp\" transform=\"translate(0,254.49333333333334)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"y3grid crisp\" transform=\"translate(0,230.41333333333336)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"y3zl zl crisp\" transform=\"translate(0,302.66333333333336)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,217.33333333333334)\" clip-path=\"url(#clip82e46ax3y3plot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M8.87,85.33V4.27H79.79V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(0, 204, 150); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(44.330000000000005,19.27)\">57.65%</text></g><g class=\"point\"><path d=\"M97.52,85.33V25.78H168.44V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(0, 204, 150); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(132.98,40.78)\">42.35%</text></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"x3tick\"><text text-anchor=\"middle\" x=\"0\" y=\"315.6666666666667\" transform=\"translate(124.33,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">wrong</text></g><g class=\"x3tick\"><text text-anchor=\"middle\" x=\"0\" y=\"315.6666666666667\" transform=\"translate(212.98,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">right</text></g></g><g class=\"yaxislayer-above\"><g class=\"y3tick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,302.66333333333336)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"y3tick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,278.58333333333337)\">20k</text></g><g class=\"y3tick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,254.49333333333334)\">40k</text></g><g class=\"y3tick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,230.41333333333336)\">60k</text></g></g><g class=\"overaxes-above\"/></g><g class=\"subplot x4y4\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x4\"/><g class=\"y4\"><path class=\"y4grid crisp\" transform=\"translate(0,268.9433333333333)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"y4grid crisp\" transform=\"translate(0,235.22333333333336)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"y4zl zl crisp\" transform=\"translate(0,302.66333333333336)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(296.70000000000005,217.33333333333334)\" clip-path=\"url(#clip82e46ax4y4plot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M8.86,85.33V4.27H79.79V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(171, 99, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(44.325,19.27)\">64.11%</text></g><g class=\"point\"><path d=\"M97.51,85.33V39.94H168.43V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(171, 99, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(132.97,54.94)\">35.89%</text></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"x4tick\"><text text-anchor=\"middle\" x=\"0\" y=\"315.6666666666667\" transform=\"translate(341.03000000000003,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">wrong</text></g><g class=\"x4tick\"><text text-anchor=\"middle\" x=\"0\" y=\"315.6666666666667\" transform=\"translate(429.68000000000006,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">right</text></g></g><g class=\"yaxislayer-above\"><g class=\"y4tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" transform=\"translate(0,302.66333333333336)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"y4tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,268.9433333333333)\">50k</text></g><g class=\"y4tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,235.22333333333336)\">100k</text></g></g><g class=\"overaxes-above\"/></g><g class=\"subplot x5y5\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x5\"/><g class=\"y5\"><path class=\"y5grid crisp\" transform=\"translate(0,382.4066666666667)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"y5grid crisp\" transform=\"translate(0,344.81666666666666)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"y5zl zl crisp\" transform=\"translate(0,419.99666666666667)\" d=\"M80,0h177.3\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(80,334.6666666666667)\" clip-path=\"url(#clip82e46ax5y5plot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M8.87,85.33V4.27H79.79V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(255, 161, 90); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(44.330000000000005,19.27)\">54.96%</text></g><g class=\"point\"><path d=\"M97.52,85.33V18.89H168.44V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(255, 161, 90); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(132.98,33.89)\">45.04%</text></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"x5tick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(124.33,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">wrong</text></g><g class=\"x5tick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(212.98,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">right</text></g></g><g class=\"yaxislayer-above\"><g class=\"y5tick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" transform=\"translate(0,419.99666666666667)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"y5tick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,382.4066666666667)\">50k</text></g><g class=\"y5tick\"><text text-anchor=\"end\" x=\"79\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,344.81666666666666)\">100k</text></g></g><g class=\"overaxes-above\"/></g><g class=\"subplot x6y6\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x6\"/><g class=\"y6\"><path class=\"y6grid crisp\" transform=\"translate(0,397.5466666666667)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"y6grid crisp\" transform=\"translate(0,375.0866666666667)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"y6grid crisp\" transform=\"translate(0,352.62666666666667)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g></g><g class=\"zerolinelayer\"><path class=\"y6zl zl crisp\" transform=\"translate(0,419.99666666666667)\" d=\"M296.70000000000005,0h177.29999999999998\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(296.70000000000005,334.6666666666667)\" clip-path=\"url(#clip82e46ax6y6plot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M8.86,85.33V4.27H79.79V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(25, 211, 243); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(44.325,19.27)\">68.27%</text></g><g class=\"point\"><path d=\"M97.51,85.33V47.65H168.43V85.33Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(25, 211, 243); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/><text class=\"bartext bartext-inside\" text-anchor=\"middle\" data-notex=\"1\" x=\"0\" y=\"0\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(68, 68, 68); fill-opacity: 1; white-space: pre;\" transform=\"translate(132.97,62.65)\">31.73%</text></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"x6tick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(341.03000000000003,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">wrong</text></g><g class=\"x6tick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(429.68000000000006,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">right</text></g></g><g class=\"yaxislayer-above\"><g class=\"y6tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" transform=\"translate(0,419.99666666666667)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"y6tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,397.5466666666667)\">50k</text></g><g class=\"y6tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,375.0866666666667)\">100k</text></g><g class=\"y6tick\"><text text-anchor=\"end\" x=\"295.70000000000005\" y=\"4.199999999999999\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(0,352.62666666666667)\">150k</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-82e46a\"><g class=\"clips\"/><clipPath id=\"legend82e46a\"><rect width=\"206\" height=\"124\" x=\"0\" y=\"0\"/></clipPath></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"legend\" pointer-events=\"all\" transform=\"translate(481.88,100)\"><rect class=\"bg\" shape-rendering=\"crispEdges\" width=\"206\" height=\"124\" x=\"0\" y=\"0\" style=\"stroke: rgb(68, 68, 68); stroke-opacity: 1; fill: rgb(255, 255, 255); fill-opacity: 1; stroke-width: 0px;\"/><g class=\"scrollbox\" transform=\"\" clip-path=\"url(#legend82e46a)\"><g class=\"groups\"><g class=\"traces\" transform=\"translate(0,14.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Type: other</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"200.53125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,33.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Type: algebrafieldquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0.5px; fill: rgb(239, 85, 59); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"200.53125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,52.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Type: 1</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0.5px; fill: rgb(0, 204, 150); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"200.53125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,71.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Type: noprobtype</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0.5px; fill: rgb(171, 99, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"200.53125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,90.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Type: radioquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0.5px; fill: rgb(255, 161, 90); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"200.53125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g><g class=\"traces\" transform=\"translate(0,109.5)\" style=\"opacity: 1;\"><text class=\"legendtext\" text-anchor=\"start\" x=\"40\" y=\"4.680000000000001\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre;\">Type: textfieldquestion</text><g class=\"layers\" style=\"opacity: 1;\"><g class=\"legendfill\"/><g class=\"legendlines\"/><g class=\"legendsymbols\"><g class=\"legendpoints\"><path class=\"legendundefined\" d=\"M6,6H-6V-6H6Z\" transform=\"translate(20,0)\" style=\"stroke-width: 0.5px; fill: rgb(25, 211, 243); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g><rect class=\"legendtoggle\" x=\"0\" y=\"-9.5\" width=\"200.53125\" height=\"19\" style=\"fill: rgb(0, 0, 0); fill-opacity: 0;\"/></g></g></g><rect class=\"scrollbar\" rx=\"20\" ry=\"3\" width=\"0\" height=\"0\" x=\"0\" y=\"0\" style=\"fill: rgb(128, 139, 164); fill-opacity: 1;\"/></g><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Percent of correct answers for top 6 problem type</text></g><g class=\"g-xtitle\"/><g class=\"g-x2title\"/><g class=\"g-x3title\"/><g class=\"g-x4title\"/><g class=\"g-x5title\"/><g class=\"g-x6title\"/><g class=\"g-ytitle\"/><g class=\"g-y2title\"/><g class=\"g-y3title\"/><g class=\"g-y4title\"/><g class=\"g-y5title\"/><g class=\"g-y6title\"/></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = ds.sort_values(['percent']).tail(6)\n",
    "\n",
    "fig = make_subplots(rows=3, cols=2)\n",
    "\n",
    "traces = [\n",
    "    go.Bar(\n",
    "        x = ['wrong', 'right'],\n",
    "        y = [\n",
    "            len(data[(data['problemType'] == item) & (data['correct'] == 0)]),\n",
    "            len(data[(data['problemType'] == item) & (data['correct'] == 1)])\n",
    "        ],\n",
    "        name = 'Type: ' + str(item),\n",
    "        text = [\n",
    "            str(round(100*len(data[(data['problemType'] == item)&(data['correct'] == 0)])/len(data[data['problemType'] == item]),2)) + '%',\n",
    "            str(round(100*len(data[(data['problemType'] == item)&(data['correct'] == 1)])/len(data[data['problemType'] == item]),2)) + '%'\n",
    "        ],\n",
    "        textposition = 'auto'\n",
    "    ) for item in ds['problemType'].unique().tolist()\n",
    "]\n",
    "\n",
    "for i in range(len(traces)):\n",
    "    fig.append_trace(\n",
    "        traces[i],\n",
    "        (i //2) + 1,\n",
    "        (i % 2) + 1\n",
    "    )\n",
    "    \n",
    "fig.update_layout(\n",
    "    title_text = 'Percent of correct answers for top 6 problem type',\n",
    ")\n",
    "\n",
    "fig.show(\"svg\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "186d03e2",
   "metadata": {},
   "source": [
    "*******\n",
    "## Sort by skills"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "6fdc1fc7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg class=\"main-svg\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"700\" height=\"500\" style=\"\" viewBox=\"0 0 700 500\"><rect x=\"0\" y=\"0\" width=\"700\" height=\"500\" style=\"fill: rgb(255, 255, 255); fill-opacity: 1;\"/><defs id=\"defs-f568b5\"><g class=\"clips\"><clipPath id=\"clipf568b5xyplot\" class=\"plotclip\"><rect width=\"342\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clipf568b5x\"><rect x=\"278\" y=\"0\" width=\"342\" height=\"500\"/></clipPath><clipPath class=\"axesclip\" id=\"clipf568b5y\"><rect x=\"0\" y=\"100\" width=\"700\" height=\"320\"/></clipPath><clipPath class=\"axesclip\" id=\"clipf568b5xy\"><rect x=\"278\" y=\"100\" width=\"342\" height=\"320\"/></clipPath></g><g class=\"gradients\"/><g class=\"patterns\"/></defs><g class=\"bglayer\"><rect class=\"bg\" x=\"278\" y=\"100\" width=\"342\" height=\"320\" style=\"fill: rgb(229, 236, 246); fill-opacity: 1; stroke-width: 0;\"/></g><g class=\"layer-below\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"cartesianlayer\"><g class=\"subplot xy\"><g class=\"layer-subplot\"><g class=\"shapelayer\"/><g class=\"imagelayer\"/></g><g class=\"gridlayer\"><g class=\"x\"><path class=\"xgrid crisp\" transform=\"translate(361.2,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(444.4,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(527.59,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/><path class=\"xgrid crisp\" transform=\"translate(610.79,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 1px;\"/></g><g class=\"y\"/></g><g class=\"zerolinelayer\"><path class=\"xzl zl crisp\" transform=\"translate(278,0)\" d=\"M0,100v320\" style=\"stroke: rgb(255, 255, 255); stroke-opacity: 1; stroke-width: 2px;\"/></g><path class=\"xlines-below\"/><path class=\"ylines-below\"/><g class=\"overlines-below\"/><g class=\"xaxislayer-below\"/><g class=\"yaxislayer-below\"/><g class=\"overaxes-below\"/><g class=\"plot\" transform=\"translate(278,100)\" clip-path=\"url(#clipf568b5xyplot)\"><g class=\"barlayer mlayer\"><g class=\"trace bars\" style=\"opacity: 1;\"><g class=\"points\"><g class=\"point\"><path d=\"M0,319.2V312.8H34.12V319.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,311.2V304.8H35.05V311.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,303.2V296.8H36.52V303.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,295.2V288.8H37.28V295.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,287.2V280.8H38.01V287.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,279.2V272.8H38.86V279.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,271.2V264.8H40.18V271.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,263.2V256.8H44.36V263.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,255.2V248.8H46.83V255.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,247.2V240.8H46.88V247.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,239.2V232.8H49.06V239.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,231.2V224.8H50.78V231.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,223.2V216.8H50.94V223.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,215.2V208.8H52.01V215.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,207.2V200.8H57.84V207.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,199.2V192.8H61.35V199.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,191.2V184.8H62.22V191.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,183.2V176.8H62.62V183.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,175.2V168.8H63.34V175.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,167.2V160.8H64.82V167.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,159.2V152.8H66.24V159.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,151.2V144.8H67.5V151.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,143.2V136.8H74.45V143.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,135.2V128.8H75.1V135.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,127.2V120.8H77.2V127.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,119.2V112.8H77.48V119.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,111.2V104.8H79.98V111.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,103.2V96.8H80.13V103.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,95.2V88.8H83.89V95.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,87.2V80.8H91.42V87.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,79.2V72.8H91.95V79.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,71.2V64.8H94.62V71.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,63.2V56.8H101.86V63.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,55.2V48.8H103.37V55.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,47.2V40.8H109.85V47.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,39.2V32.8H141.29V39.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,31.2V24.8H142.72V31.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,23.2V16.8H187.31V23.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,15.2V8.8H304.43V15.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g><g class=\"point\"><path d=\"M0,7.2V0.8H324.9V7.2Z\" style=\"vector-effect: non-scaling-stroke; opacity: 1; stroke-width: 0.5px; fill: rgb(99, 110, 250); fill-opacity: 1; stroke: rgb(229, 236, 246); stroke-opacity: 1;\"/></g></g></g></g></g><g class=\"overplot\"/><path class=\"xlines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><path class=\"ylines-above crisp\" d=\"M0,0\" style=\"fill: none;\"/><g class=\"overlines-above\"/><g class=\"xaxislayer-above\"><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" transform=\"translate(278,0)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">0</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(361.2,0)\">20k</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(444.4,0)\">40k</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(527.59,0)\">60k</text></g><g class=\"xtick\"><text text-anchor=\"middle\" x=\"0\" y=\"433\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\" transform=\"translate(610.79,0)\">80k</text></g></g><g class=\"yaxislayer-above\"><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,416)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">finding-percents-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,400)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">least-common-multiple-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,384)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">linear-area-volume-conversion-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,368)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">scientific-notation-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,352)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">supplementary-angles-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,336)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">equivalent-fractions-decimals-percents-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,320)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">addition-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,304)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">algebraic-manipulation-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,288)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">mean-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,272)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">subtraction-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,256)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">combinatorics-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,240)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">evaluating-functions-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,224)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">discount-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,208)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">substitution-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,192)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">interpreting-linear-equations-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,176)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">pythagorean-theorem-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,160)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">square-root-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,144)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">multiplication-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,128)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">area-</text></g><g class=\"ytick\"><text text-anchor=\"end\" x=\"277\" y=\"4.199999999999999\" transform=\"translate(0,112)\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 12px; fill: rgb(42, 63, 95); fill-opacity: 1; white-space: pre; opacity: 1;\">probability-</text></g></g><g class=\"overaxes-above\"/></g></g><g class=\"polarlayer\"/><g class=\"ternarylayer\"/><g class=\"geolayer\"/><g class=\"funnelarealayer\"/><g class=\"pielayer\"/><g class=\"iciclelayer\"/><g class=\"treemaplayer\"/><g class=\"sunburstlayer\"/><g class=\"glimages\"/><defs id=\"topdefs-f568b5\"><g class=\"clips\"/></defs><g class=\"layer-above\"><g class=\"imagelayer\"/><g class=\"shapelayer\"/></g><g class=\"infolayer\"><g class=\"g-gtitle\"><text class=\"gtitle\" x=\"35\" y=\"50\" text-anchor=\"start\" dy=\"0em\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 17px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">Top 40 useful skills</text></g><g class=\"g-xtitle\"><text class=\"xtitle\" x=\"449\" y=\"460.3\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">count</text></g><g class=\"g-ytitle\" transform=\"translate(2.4560546875,0)\"><text class=\"ytitle\" transform=\"rotate(-90,11.543749999999989,260)\" x=\"11.543749999999989\" y=\"260\" text-anchor=\"middle\" style=\"font-family: 'Open Sans', verdana, arial, sans-serif; font-size: 14px; fill: rgb(42, 63, 95); opacity: 1; font-weight: normal; white-space: pre;\">skill</text></g></g></svg>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds = data['skill'].dropna() # There are less NaNs in 'skill_id' column than 'skill_name' column.\n",
    "ds = ds.value_counts().reset_index()\n",
    "\n",
    "ds.columns = [\n",
    "    'skill',\n",
    "    'count'\n",
    "]\n",
    "\n",
    "ds['skill'] = ds['skill'].astype(str) + '-'\n",
    "ds = ds.sort_values(['count']).tail(40)\n",
    "\n",
    "fig = px.bar(\n",
    "    ds,\n",
    "    x = 'count',\n",
    "    y = 'skill',\n",
    "    orientation = 'h',\n",
    "    title = 'Top 40 useful skills'\n",
    ")\n",
    "\n",
    "fig.show(\"svg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19916187",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
